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FOREWORD 


This  Technical  Report  is  the  result  of  a work  effort  performed  by 
the  Digital  Applications  Group  of  the  Crew  Systems  Integration 
Branch  (FGR),  Air  Force  Flight  Dynamics  Laboratory,  Wright-Patterson 
Air  Force  Base,  Ohio.  Major  Robert  Bateman  is  the  group  leader  and 
Dr.  John  Reising  is  responsible  for  human  factors.  Mr.  Emmett  Herron 
of  the  Bunker  Ramo  Corporation  is  tasked  with  providing  pilot  inputs  to 
the  work  efforts,  and  Ms.  Gloria  Calhoun  of  the  same  company  is  tasked 
with  statistical  and  experimental  design  inputs.  The  objective  of  this 
effort  was  to  evaluate  the  use  of  two  specific  multifunction  keyboards 
within  the  cockpit.  The  hardware  was  provided  by  AF  Avionics  Laboratory. 

The  Bunker  Ramo  portion  of  the  work  effort  was  performed  under  USAF 
Contract  Numbers  F3361 5-73C-0391  and  F3361 5-76C-0013.  The  contract  was 
initiated  under  Project  Number  6190,  "Control-Display  for  Air  Force 
Aircraft  and  Aerospace  Vehicles"  which  is  managed  by  Mr.  J.  H.  Kearns,  III, 
as  Project  Engineer  and  Principal  Scientist  for  the  Crew  Systems 
Integration  Branch  (AFFDL/FGR)  Flight  Control  Division,  Air  Force  Flight 
Dynamics  Laboratory. 


This  effort  was  performed  as  part  of  the  Digital  Avionics  Infor 
mation  System  (DAIS)  Advanced  Development  Program  under  Work  Unit 
20490202,  and  was  performed  in  support  of  the  Air  Force  Avionics 
Laboratory  Work  Unit  20030624. 

This  report  includes  work  performed  between  3 July  1974  and 
31  December  1975. 
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GLOSSARY  OF  TERMS 

AAE  - average  absolute  error  - see  Appendix  F. 

AE  - average  error  - see  Appendix  F. 

COC  6600  - Control  Data  Corporation  general  purpose  computer. 

DEC  10  - Digital  Equipment  general  purpose  computer. 

DEDICATED  SWITCH  - single  switch  capable  of  performing  only  one  function. 

DIGIT/MODE  PANEL  - panel  with  seventeen  dedicated  switches  used  in  the 
present  study  for  data  entry  and  mode  selection. 

FIGURE  OF  MERIT  - statistical  procedure  used  in  data  analysis  - see 
Appendix  F. 

FLIGHT  PLAN  - AF  Form  70  specifying  radio  frequencies  and  waypoint 
coordinates. 

FLYING  TASK  - maintaining  ground  speed  and  altitude  parameters  and  keeping 
the  flight  director  centered  on  the  Vertical  Situation  Display. 

FOM  - see  Figure  of  Merit. 

HORIZONTAL  SITUATION  DISPLAY  - cathode  ray  tube  used  to  present  navigation 
information. 

HSO  - see  Horizontal  Situation  Display. 

KEYBOARD  TASK  • operating  the  keyboards  during  coanunicatioo  changes  and 
navigation  updates. 

LOGIC  LEVELS  - means  by  which  pilots  selected  and  executed  tasks;  each 
change  of  switch  function  constituted  a single  logic  level. 

NANOVA  - see  Multivariate  Analysis  of  Variance. 

NFK  • see  Multifunction  Keyboard. 

MPO  - see  Multipurpose  Display. 

NULTIFUNCTIOM  CONTROLS  - several  multifunction  switches  on  a single  panel. 

MULTIFUNCTION  KEYBOARDS  • several  multifunction  push  button  type  switches 
on  a single  panel. 

MULTIPURPOSE  DISPLAY  - cathode  ray  tube  used  to  present  various  types  of 
status  information. 

MULTIFiWCTION  SWITCH  - a switch  whose  function  changes,  depending  upon 
the  task  being  performed  by  the  operator. 
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GLOSSARY  OF  TERMS  (CONTINUED) 

MULTIVARIATE  ANALYSIS  OF  VARIANCE  - statistical  procedure  used  in  data 
analysis  - see  Appendix  F. 

POP  11/45  - Digital  Equipment  Corporation  general  purpose  minicomputer. 

PLASMA  PANEL  HARDWARE  - MFK  hardware  in  which  the  legend  on  a display 
adjacent  to  the  switch  changes  according  to  the  function  the  switch 
is  serving  at  the  time. 

PROJECTION  SWITCH  HARDWARE  - MFK  hardware  made  up  of  switches  having  the 
capability  to  display  different  legends  by  selectively  projecting 
different  parts  of  a film  strip  onto  the  switch  front  surface. 

RAMTEK  RASTER  SYMBOL  GENERATOR  - a display  system  which  converts  computer 
generated  alphanumeric  and  graphic  display  information  into  industry 
compatible  video  signals. 

RANDOMIZED  BLOCK  FACTORIAL  DESIGN  > experimental  design  in  which  each 
subject  receives  all  combinations  of  experimental  conditions.  The 
order  of  administration  of  the  treatment  combinations  was  randomized 
independently  for  each  subject. 

RMS  • root  mean  square  - see  Appendix  F. 

SO  • standard  deviation  • see  Appendix  F. 

STEPWISE  DISCRIMINANT  FUNCTION  ANALYSIS  - statistical  procedure  used  in 
data  analysis  - see  Appendix  F. 

VERTICM.  situation  display  - cathode  ray  tube  used  to  present  flight 
information. 

VSO  • see  Vertical  Situation  Display. 
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SUMMARY 

Multifunction  keyboards  (MFKs)  have  been  designed  to  integrate  the 
many  dedicated  control  functions  found  in  present  day  cockpits  into  a 
more  efficient  arrangement.  The  purpose  of  this  study  was  to  examine 
pilot  performance  changes  while  operating  MFKs  during  simulated  flight. 
Performance  in  terms  of  maintaining  flight  parameters  and  operating  the 
keyboards  was  recorded  during  communication  changes  and  navigation  updates. 
The  specific  test  objectives  were:  (1)  evaluate  and  compare  two  MFK 

hardware  types  - plasma  panel  and  projection  switches;  (2)  evaluate  four 
different  arrangements  of  the  task  steps  or  logic  levels  across  keyboards; 
(3)  assess  the  impact  of  both  a center  and  side  control  stick  location 
on  MFK  operation;  and  (4)  evaluate  the  operation  of  a right  console 
backup  keyboard  when  a primary  keyboard  fails. 

One  conclusion  reached  as  a result  of  this  study  was  that  operations 
other  than  digit  entry  should  be  consolidated  on  a single  keyboard. 
Furthermore,  the  study  showed  that  performance  was  better  when  the  digits 
were  entered  on  a separate  panel.  It  remains  to  be  determined  whether 
the  better  performance  was  due  to  the  dedicated  switches  or  to  the 
optimized  number  arrangement.  The  study  also  indicated  that  so  long  as 
MFK  operation  is  not  physically  inhibited  by  the  center  or  side  location 
of  the  control  stick,  performance  is  not  affected.  Concerning  the  MFK 
hardware  types  evaluated,  performance  was  generally  better  with  the 
projection  switch  MFK  compared  to  the  plasma  panel  MFK.  Operation  of 
both  types  of  MFKs  on  the  front  panel  and  right  console  locations  is 
discussed.  Because  of  design  faults  in  the  two  specific  keyboards 
provided  by  AF  Avionics  Laboratory,  no  attempt  should  be  made  to 
generalize  the  results  of  this  evaluation  to  other  keyboards.  The  need 
■for  further  research  is  indicated. 
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SECTION  I 
INTRODUCTION 


1 . BACKGROUND 

During  the  1960's,  there  was  a significant  increase  in  the  application 
of  digital  computers  to  the  avionics  subsystem  of  both  military  and 
civilian  aircraft.  The  ability  to  miniaturize  digital  circuits  through 
the  use  of  large  scale  Integration  has  enabled  the  avionics  subsystem 
designer  to  take  advantage  of  the  flexibility  of  the  digital  computer 
without  paying  the  weight  penalty  associated  with  earlier  versions  of 
general  purpose  digital  computers.  As  the  "digital  airplane,"  i.e., 
one  in  which  all  the  subsystems  are  managed  by  digital  processors, 
approaches  reality,  there  are  some  significant  impacts  on  the  cockpit. 

This  digital  capability  allows  the  pilot  access  to  a great  deal  of 
information.  However,  matching  this  expanded  ability  to  process  and 
manipulate  information  with  the  conventional  approach  of  dedicated 
instruments  and  switches  requires  so  many  displays  and  control  devices 
that  cockpit  size  prevents  the  designer  from  getting  the  full  value  of 
the  computer.  In  fact,  the  continued  use  of  such  controls,  displays, 
and  switches  will  result  in  a cockpit  that  is  overloaded  with  dedicated 
devices  and  aircrews  will  pay  a high  workload  penalty  for  the  luxury  of 
an  on-board  digital  computer.  As  the  information  continues  to  increase, 
it  will  not  be  physically  possible  to  provide  for  the  multitude  of  display 
options  with  a dedicated  display  for  each. 

This  increase  in  information  to  be  displayed  has  led  to  the  "time- 
shared"  concept,  in  which  the  information  presented  on  the  display 
changes  as  a function  of  information  requirements.  The  time-shared 
concept  can  also  be  extended  to  switches.  The  inherent  flexibility  of 
the  digital  computer  allows  it  to  change  the  meaning  of  switches  as  a 
function  of  mission  requirements.  In  this  way,  the  digital  computer 
not  only  can  simplify  the  pilot's  task  of  performing  routine  functions, 
but  also  can  optimize  the  information  presentation  end  reduce  the 
number  of  switches  needed.  Realization  of  the  full  power  of  the  digital 
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computer  depends  upon  the  ability  of  the  pilot  to  interpret  the  different 
display  formats  and  to  properly  select  the  correct  multifunction  switch. 

While  the  digital  processors  can  simplify  routine  control  functions  and 
perform  computations  for  the  pilot,  the  system  design  must  allow  the  pilot 
to  exercise  judgment  and  be  able  to  control,  in  detail,  all  system 
components. 

The  Air  Force  has  conducted  a series  of  research  efforts  to  examine 
the  cockpit  implication  of  digital  computers  (References  1,  2,  3,  4). 

These  efforts  have  centered  around  the  engineering  problems  involved  in 
integrating  the  sensors,  processors,  displays/controls  in  the  digital 
aircraft,  and  the  human  factors  problems  involved  in  piloting  this  air- 
craft. The  human  factors  research  initially  emphasized  the  electro- 
optical  display  formats,  but  early  in  these  research  efforts  it  became 
clear  that  the  multifunction  controls  were  equally  as  important,  if  not 
more  important,  than  the  displays  in  determining  the  success  of  the 
digital  aircraft  cockpit  (Reference  5). 

A multifunction  control  is  a panel  made  up  of  several  multifunction 
switches;  each  switch  is  capable  of  performing  more  than  one  function. 

If  the  switches  are  push  buttons  or  keys,  the  device  Is  called  a multi- 
function keyboard  (MFK).  Each  switch  Is  capable  of  Inputting  different 
bits  of  information  due  to  the  implementation  of  a logic  network.  Thus, 
it  is  essential  that  the  pilot  know  the  significance  of  each  switch 
actuation.  To  accomplish  this,  the  legend  for  each  switch  must  be 
appropriate  to  the  function  it  is  serving  at  the  time.  Projection 
switch  hardware  changes  a legend  on  the  switch  itself.  Other  mechanizations, 
e.g.,  plasma  panels,  change  a legend  on  a display  surface  adjacent  to 
the  switch.  No  matter  what  the  type  of  mechanization,  the  essential 
features  of  the  HFK  remain  the  same.  Dedicated,  single  purpose  master 
switches  enable  the  pilot  to  establish  an  initial  set  of  capabilities  for 
the  multifunction  switches.  Then,  the  multifunction  switches  allow  the 
pilot  to  perform  specific  operations.  For  example,  a plasma  panel 
version  of  an  MFK  is  shown  in  Figure  1.  Across  the  top  of  the  display 
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"igure  1.  An  Example  of  One  Type  of  Plasma  Panel  MFK 

surface  are  nine  dedicated  Raster  switches.  The  multifunction  switches 
ere  mounted  In  columns  on  the  left  and  right  portions  of  the  bezel  arid 
have  no  legends  on  them.  Each  legend  appears  on  the  plasma  panel  next 
to  the  switch.  The  number  of  these  legends  for  each  switch  Is  limited 
only  by  the  memory  In  the  digital  computer.  In  Figure  1,  the  master 
switch  labeled  COMM  (for  communkationsi  has  been  selected.  Therefore, 
the  legends  appearing  next  to  the  multlfuiictlon  switches  Indicate  a 
variety  of  communication  radios  which  the  pilot  may  wish  to  control. 

The  next  step  would  be  to  select  the  s^^clflc  radio  to  be  operated. 

This  selection  would  change  the  legends  to  appropriate  titles  for  the 
multifunction  switches  and  would  allow  the  pilot  to  turn  on  the  radio, 
change  frequency  or  whatever.  Each  c.«ange  of  switch  function  Is 
called  a “logic  level." 

The  MFK  provides  tremendous  freedom  for  the  cockpit  designer  In  that 
he  can  allocate  a number  of  functions  to  a single  control  panel  and,  thus, 
reduce  the  number  of  control  heads  and  switches  In  the  cockpit  (Reference  6). 
This  design  helps  the  pilot  by  providing  a single,  easily  reachable 
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keyboard  with  which  he  may  control  several  different  systems.  As  a 
result,  cockpit  clutter  is  reduced,  panels  in  hard  to  reach  places  are 
eliminated,  and  switch  actions  become  the  same,  i.e.,  push  buttons. 

However,  there  are  some  issues  to  consider  in  the  design  of  the  MFK. 
For  example,  as  more  and  more  functions  are  located  on  a single  control 
head,  time  sharing  problems  arise.  The  communications  system  provides  a 
convenient  illustration.  Let  us  suppose  the  pilot  were  changing  a UHF 
radio  frequency  immediately  after  takeoff  in  response  to  a request  from 
departure  control.  A change  in  the  transponder  code  can  also  be  part 
of  this  request.  Since  the  pilot  doesn't  have  separate  control  heads  for 
each  of  these  radios,  he  must  initiate  and  proceed  through  the  COMM/UHF 
sequence  to  change  frequency  and  then  go  back  and  do  the  same  for  the 
COMM/IFF  sequence  to  change  code.  A worst  case  would  be  generated  when, 
halfway  through  one  task,  a pilot  was  required  to  initiate  another  task. 
Problems  may  also  arise  if  operators  become  "lost"  in  a maze  of  logic 
trees  or  forget  where  they  are.  The  extent  of  these  problems  remains  to 
be  determined.  The  crew  station  designer  must  be  fully  aware  of  these 
problems  when  designing  the  MFKs.  One  solution  to  the  single  panel 
problem  is  to  use  two  identical  MFKs,  thereby  providing  the  capability  to 
start  a task  on  one  MFK;  stop  that  task  to  initiate  and  complete  a task 
on  the  second  MFK;  and  then  return  to  the  first  MFK  and  complete  the  first 
task.  Such  a system  would  still  occupy  less  space  than  the  many  control 
panels  that  are  currently  used.  The  use  of  two  MFKs  by  allocating  half 
of  the  functions  to  each  control  panel  would  also  be  a solution. 

Another  issue  to  be  considered  when  discussing  MFKs  is  redundancy. 
What  happens  if  the  CRT  becomes  Inoperative?  What  if  a master  function 
switch  fails  in  either  open  or  closed  position?  What  if  a multifunction 
switch  breaks?  Such  possibilities  make  the  inclusion  of  a second  MFK 
attractive.  The  inclusion  of  an  identical  second  MFK  may  be  the  best 
solution  (given  that  space  constraints  or  computer  limitations  do  not 
rule  it  out)  since  complete  redundancy  is  achieved  and  the  pilot  can 
operate  functions  separately  on  each  keyboard.  The  final  solution  as  to 
which  back  up  capability  is  best  depends  upon,  among  other  things,  the 
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type  of  control  stick  used.  Control  stick  location  impacts  the  pilots' 
reach  envelope,  thus  the  placement  of  a second  MFK  may  vary  if  a side 
stick  is  used  instead  of  the  conventional  center  stick. 

2.  PURPOSE 

The  MFK  has  been  designed  to  integrate  the  many  dedicated  control 
functions  found  in  present  day  cockpits  into  a more  efficient  arrangement. 
The  purpose  of  this  study  was  to  examine  pilot  performance  changes  while 
operating  MFKs  during  simulated  flight.  The  following  specific  factors 
related  to  MFK  operation  were  investigated; 

a.  MFK  Hardware  Type 

Three  types  of  keyboards  were  used.  The  main  thrust  of  the 
investigation  was  to  compare  two  of  these  (projection  switch  MFK  with  a 
plasma  panel  MFK).  For  each  task,  one  of  the  MFKs  was  mounted  on  the 
front  panel  and  the  other  MFK  was  mounted  on  the  right  console.  Both 
MFKs  were  evaluated  in  both  locations  (Figures  2 and  3).  Each  of  these 
two  keyboards  was  used  in  conjunction  with  a dedicated  third  keyboard 
for  some  tasks.  This  dedicated  keyboard  included  switches  for  mode 
selection  and  for  digit  entry,  hence,  it  was  referred  to  as  a Digit/Mode 
Panel.  It  was  always  located  on  the  left  console  (see  Figures  2 and  3). 

b.  Logic  Level  Arrangements 

Each  task,  whether  a communication  change  or  a navigation 
update,  required  a four  step  operating  sequence.  Each  step  in  these 
operating  sequences  is  called  a logic  level.  For  example,  in  the  present 
study,  the  pilot  had  to  go  through  the  following  four  steps  or  logic 
levels  to  change  a UHF  radio  frequency: 

Step  1 - select  the  communication  function  from  all  other  functions 
on  the  keyboard. 

Step  2 - select  the  UHF  radio  from  among  the  other  radios  on  board 
the  aircraft. 

Step  3 - select  the  frequency  change  function  from  among  the  other 
functions  of  the  UHF  radio. 

Step  4 - enter  the  appropriate  frequency. 

■ 5 


Figure  2.  Cockpit  Configuration  with  Front  Panel  Projection 
Switch  NFK  and  Right  Console  Plasna  Panel  NFK 
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Figure  3.  Cockpit  Configuration  with  Front  Panel  Plasma  Panel 
NFK  and  Right  Console  Projection  Switch  NFK 
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Each  MFK  had  the  capability  for  all  four  levels  of  systems  control  and 
information,  whereas  the  Digit/Mode  Panel,  had  only  the  capability  to 
function  for  logic  levels  1 and/or  4.  This  study  examined  logic  level 
arrangements  in  terms  of  whether  operations  of  the  four  logic  levels 
should  be  performed  on  one  keyboard  only  (one  of  the  MFKs)  or  two  keyboards 
(divided  between  one  of  the  MFKs  and  the  Digit/Mode  Panel).  Figure  4 
shows  the  four  different  logic  level  arrangements,  lettered  A through  D, 
which  were  investigated. 

c.  Control  Stick  Location 

Another  factor  investigated  in  relation  to  MFK  operation  was 
the  effect  of  both  a center  and  side  control  stick  location  on  the 
operation  of  the  MFKs  (Figure  5).  The  distinction  should  be  made  that 
it  was  not  the  intent  of  this  study  to  evaluate  differences  in  stick 
location,  but  rather  the  effects  of  stick  location  on  MFK  operation. 

The  MFK  being  evaluated  in  the  front  location  was  designated  as  "primary" 
for  a task.  When  failures  of  the  primary  MFK  were  introduced,  the 
other  MFK,  mounted  on  the  right  console,  was  used  as  a "backup".  It  was 
expected  that  a center  stick  would  tend  to  interfere  with  the  primary 
MFK  and  that  a side  stick  would  tend  to  interfere  with  the  backup  MFK 
(Figures  6 and  7). 

d.  Degraded  Mode  Performance  Between  MFKs 

One  crucial  drawback  to  the  MFK  is  the  loss  of  capability  with 
keyboard  failure.  This  study  dealt  specifically  with  this  problem  in 
that  it  studied  the  operation  of  backup  NFKs  to  be  used  when  the  primary 
keyboard  fails.  Operation  during  normal  trades  involved  either  the  front 
instrument  panel  MFK  or  the  front  instrunent  panel  MFK  and  the  Oigit/Hode 
Panel.  Failed  modes  involved  operation  of  either  the  right  console  MFK 
or  the  right  console  MFK  and  the  Digit/Mode  Panel.  Failures  were 
initiated  only  between  task  events.  During  a failed  mode,  either  the 
front  panel  MFK  or  the  Digit/Hode  Panel,  became  inoperative.  The  logic 
levels  (Figure  4)  that  had  been  on  that  keyboard  then  became  operable  on 
the  right  console  backup  MFK  (Figure  8).  Both  the  projection  switch  and 
plasma  panel  type  MFK  were  examined  in  the  right  console  location  during 
failed  conditions. 
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e.  Additional  Cockpit  Design  Factors 

In  addition  to  obtaining  subjective  evaluations  on  the  four  areas 
just  discussed,  subjective  evaluations  were  also  obtained  on  the  display 
formats,  the  use  of  a pre-entry  readout,  and  control  actions  required  to 
correct  erroneous  entries. 


NORMAL  CONDITION 


Figure  4.  Four  Logic  Level  Arrangemnts  for  Perfoming  Steps 
In  the  Normal  Operating  Sequence 
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SECTION  II 


APPARATUS 


A two-place,  side-by-side  cockpit  simulator  of  F-lll  dimensions,  was 
fabricated  to  accomodate  the  elect'"o-optical  displays  and  MFKs.  The 
cockpit  layout  is  shown  in  Figure  9.  The  subject  pilot  was  seated  in  the 
right  side  of  the  cockpit  while  the  left  seat  was  occupied  by  an  experi- 
menter. The  controls  and  displays  for  the  left  seat  were  not  activated. 


Figure  9.  Cockpit  Simulator  Used  in  the  MFK  Evaluation 
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1.  KEYBOARD  CONFIGURATICN 
a.  Description 

Two  types  of  HFKs  and  one  dedicated  keyboard  were  utilized  in 
the  present  study.  They  were  as  follows: 

(1)  Projection  Switches 

One  MFK  consisted  of  sixteen  push  button  projection  switches 
{Figure  10).  Each  switch  had  twelve  possible  legends.  The  legends  were 
programmed  to  inform  the  pilot  of  the  four  levels  of  systems  control 
available.  Only  those  switches  displaying  information  were  operable. 


Figure  10.  Projection  Switch  MFK 
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(2)  Plasma  Panel 

The  other  MFK  utilized  a plasma  panel  with  sixteen 
peripheral  push-button  switches  (Figure  11).  Each  switch  was  associated 
with  a legend  located  on  the  plasma  panel.  Due  to  the  relative  difference 
in  sizes  of  the  switches  and  plasma  panel,  the  legends  were  not  directly 
adjacent  and  in  lino  with  the  corresponding  switches.  Therefore,  each 


switch  was  associated  with  the  appropriate  legend  by  a white  line.  The 
switches  were  operable  only  when  information  was  displayed  adjacent  to 
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(3)  Digit/Mode  Panel 

The  digit/mode  panel  consisted  of  seventeen  dedicated  push 
button  switches.  Twelve  of  the  keys  served  as  a data  entry  panel;  five 
served  as  mode  select  keys.  The  five  switches  were  backlighted  to 
indicate  mode  selected  (green-selected,  white-not  selected). 

b.  Location 

The  dedicated  digit/mode  panel  was  mounted  forward  of  the 
throttle.  Each  type  of  MFK  was  mounted  on  the  front  panel  during  half 
of  the  flights  and  on  the  right  console  during  the  other  flights.  Thus, 
the  operation  of  each  MFK  type,  both  as  a primary  (front  instrument  panel) 
keyboard  and  as  a backup  (right  console)  keyboard,  could  be  examined. 

c.  Keyboard  Logic  Levels 

As  previously  mentioned,  logic  levels  were  the  means  by  which 
the  pilot  selected  and  executed  a particular  task.  Four  logic  levels 
were  required  for  the  communication  (COMM)  and  navigation  (NAV)  tasks 
performed  by  the  pilot  in  this  study.  During  the  first  logic  level 
step,  the  pilot  selected  either  COMM  or  NAV  from  the  five  available  modes. 
Activation  of  the  mode  selection  switch  brought  up  logic  level  2 under  that 
mode.  Activation  of  a control  at  logic  level  2 changed  the  panel  to 
logic  level  3 and  presented  options  appropriate  to  that  task.  Activation 
of  a control  at  level  3 enabled  data  entry  at  logic  level  4.  Figures  12 
and  13  show  a typical  communication  change  sequence,  legends  used,  and 
legend  location  for  the  projection  switches  and  plasma  panel  MfKs, 
respectively.  Similarly,  Figures  14  and  15  illustrate  the  logic  level 
steps,  legends  used,  and  legend  location  for  each  MFK  type,  to  complete 
a navigation  update.  It  should  be  noted  that  the  selection  of  legend 
location  was  made  by  AF  Avionics  laboratory  and  hardware/software  con- 
straints made  changes  impractical. 
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Figure  12.  Projection  Switch  MFK  Communication  Change  Sequence 
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Figure  13.  Plasma  Panel  NFK  Comnuntcatlon  Change  Sequence 
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Figure  U.  Projection  Switch  MFK  Navigation  Update  Sequence 
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Figure  15.  Plasm  Panel  NFK  Navigation  Update  Sequence 
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2.  COCKPIT  CONFIGURATION 

Four  electro-optical  displays  were  used  in  the  present  study  to 
provide  information  to  the  pilot  (Figure  9).  The  Vertical  Situation 
Display  (VSD)  was  presented  on  a color  CRT  and  was  essentially  an 
electro-optical  Attitude  Director  Indicator  (ADI)  (Figure  16).  The 
Horizontal  Situation  Display  (HSD)  was  presented  on  a nine-inch  diagonal 
monochrome  CRT  and  consisted  of  a representation  of  the  route  of  flight 
(Figure  17).  Two  Multipurpose  Displays  (MPDs)  were  used  to  provide  either 
communications  or  navigation  data  on  the  left,  and  either  engine  data 
or  keyboard  failure  data  on  the  right  (Figures  18  through  21).  For  a 
more  complete  description,  see  Appendix  A. 

Other  controls  included  (a)  flight  mode  select  panel--only  the  cruise 
mode  was  used  in  this  study,  (b)  landing  gear  control  panel— landing  gear 
handle  was  not  operational— spaed  brakes  and  flaps  were  operational,  and 
(c)  pitch  indication  zeroing  switch— activation  of  this  blue-lighted 
push  button  switch  aligned  the  horizon  line  with  the  aircraft  symbol. 

Thrust  was  controlled  by  a left-side,  slide-control  throttle.  Bank 
and  pitch  commands  were  input  either  by  means  of  a center  stick  mounted 
on  the  floor  or  a side  stick  mounted  on  the  right  console.  The  side 
stick  configuration  included  an  armrest.  Both  center  and  side  stick  had 
conventional  trim  buttons  and  a microphone  button. 

3.  EXPERIMENTER'S  CONSOLE  AND  SIHUUTOR  FACILITIES 

These  items  are  described  in  detail  in  Appendix  6.  They  were 
designed  to  allow  the  experimenter  to  initiate  tasks  and  control  failures 
in  a realistic  fashion,  yet  allow  automation  of  the  test  configuration 
details. 
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Figure  20.  Multipurpose  Display  (MPD)  with  Engine  Status 
Format 


Figure  21.  Multipurpose  Display  (NPO)  with  MFK  Failure 
Format 
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SECTION  III 
TEST  METHOD/APPROACH 

1.  TEST  OBJECTIVES 

The  purpose  of  this  study  was  to  examine  pilot  performance  changes 
while  operating  MFKs  during  simulated  flight.  The  specific  test 
objectives  were:  (1)  evaluate  and  compare  the  two  MFK  hardware  types  - 
plasma  panel  and  projection  switches;  (2)  evaluate  four  logic  level 
arrangements  in  terms  of  whether  the  logic  levels  should  be  located  on 
one  keyboard  (MFK  only)  or  two  keyboards  (MFK  and  Oigit/Mode  Panel); 

(3)  assess  the  impact  of  control  stick  location  on  MFK  operation;  and 

(4)  evaluate  the  performance  of  a right  console,  backup  keyboard  when 
a primary  keyboard  fails.  The  test  design  provided  for  analyses  of: 

(1)  several  objective  performance  measures  for  four  flight  parameters  and 

(2)  two  objective  performance  measures  for  keyboard  operation.  Question- 
naire data  was  also  obtained  on  these  same  factors. 

2.  TEST  CONFIGURATION 

A Randomized  Block  Factorial  Design  (Reference  7)  was  used  in  this 
study.  The  design  involved  the  following  three  Independent  variables: 

(1)  two  locations  of  the  control  stick  (center  and  side  location); 

(2)  two  types  of  keyboard  hardtn»re  (plasma  panel  and  projection 
switches);  and  (3)  four  different  logic  level  arrangements.  Each  of  the 
nine  subject  pilots  was  scheduled  to  fly  twenty-four  test  flights. 

Sixteen  of  these  flights  reguired  failed  mode  operations  in  order  to 
evaluate  use  of  the  backup  keyboard.  The  remaining  eight  flights  were 
scheduled  without  failures.  The  normal  and  failed  mode  flights  were 
randomly  distributed  for  each  pilot  so  that  the  pilots  were  less  likely 
to  anticipate  failures.  Specific  task  order  and  route  of  flight  were 
independently  randomized  for  each  subject  pilot.  Hardware  location, 
logic  level  arrangement,  and  stick  location  were  also  independently 
randomized  and  balanced  for  each  pilot.  In  order  to  reduce  changes  of 
hardware  components,  each  pilot  flew  the  twenty-four  test  flights  in 
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groups  each  consisting  of  four  or  five  consecutive  flights  with  the  same 
keyboard  and  control  stick  configuration.  The  order  in  which  each  pilot 
was  scheduled  to  fly  these  groups  of  flights  was  random.  (See  Appendix  C 
for  daily  test  schedules.) 

3.  TEST  SUBJECTS 

A total  of  nine  pilots  served  as  volunteer  subjects  in  this 
experiment.  Seven  were  current  pilots  from  the  4950th  Test  Wing  of  the 
Aeronautical  Systems  Division  located  at  Wright-Patterson  AFB,  while  the 
other  two,  while  not  currently  assigned  flying  duties,  had  extensive 
flying  experience.  The  subject  pilots  had  an  average  age  of  29  years 
and  an  average  of  1,805  flying  hours. 

4.  TEST  PROCEDURE 

a.  Pilot  Briefing 

During  the  first  two  hour  session,  each  pilot  was  given  a fifteen 
minute  briefing  regarding  the  purpose  of  the  study.  Included  in  the 
briefing  was  a discussion  concerning  an  advanced  "digital"  airplane 
cockpit  and  specific  explanations  of  the  controls  and  displays.  After 
answering  any  questions  the  pilot  might  have  had,  the  experimenter 
took  the  pilot  to  the  simulator  for  an  additional  briefing. 

b.  Cockpit  Briefing 

A structured  outline  was  followed  during  the  cockpit  briefing  to 
standardize  training  procedures,  thus,  ensuring  that  each  pilot  received 
the  same  information  and  the  same  opportunity  for  cockpit  familiarization. 
The  thirty  minute  briefing  included  the  following:  (a)  an  explanation  of 
location  and  types  of  keyboards  and  displays;  (b)  the  detailed  operation 
of  the  keyboards  in  respect  to  the  logic  level  arrangements  during  both 
normal  and  failed  modes  of  operation;  (c)  and  specific  navigation/ 
communication  tasks  each  pilot  would  be  required  to  perform.  In  addition, 
the  display  formats  and  operation  of  cockpit  controls  were  explained.  The 
pilot  was  also  required  to  complete  some  tasks  involving  use  of  the 
keyboard  prior  to  the  actual  training  flights. 
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c.  Training  Flights 

Following  the  pilot's  familiarization  with  the  cockpit  simulator, 
four  training  flights,  one  with  each  logic  level  arrangement,  were 
conducted  in  order  to  give  the  pilot  experience  with  the  handling  qualities 
of  the  simulator  and  operational  procedures  of  the  test  conditions. 

During  the  flights,  the  pilot  completed  communication  and  navigation 
tasks  under  normal  conditions  using  the  primary  keyboard(s)  and  under 
failed  conditions  using  a right  console  keyboard.  Observation  of  the 
pilots'  performance  revealed  that  forty  minutes  of  training  was  adequate 
to  enable  them  to  control  the  simulator  and  operate  the  keyboards.  Four 
of  the  pilots  were  selected  randomly  and  trained  with  the  projection 
switches  as  the  primary  keyboard;  the  other  five  were  trained  with  the 
plasma  panel  as  the  primary  keyboard.  Similarly,  four  of  the  pilots 
were  selected  in  a separate  random  process  and  trained  with  the  control 
stick  in  a center  location,  while  the  others  were  trained  with  the  control 
stick  in  a side  location.  The  alternative  locations  of  the  keyboard 
and  control  stick  were  pointed  out  to  the  pilot. 

d.  Test  Flights 

At  the  initialization  of  each  flight,  the  displays  were  in  the 
following  configuration:  (a)  VSO  - Flight  parameters  were  appropriate 
to  that  of  level  flight  in  a cruise  mode  with  an  altitude  of  20,000  feet 
and  indicated  airspeed  of  301  knots,  (b)  HSD  - Aircraft  position  was 
approximately  seven  miles  short  of  waypoint  1 (Figure  17);  the  heading 
was  the  same  as  that  for  the  first  leg.  The  ground  speed  was  at 
420  knots,  (c)  Left  HPO  • Coonunl cations  status  format  was  displayed. 

(d)  Right  HPO  - Engine  status  format  was  displayed.  The  SENSORS  mode 
at  logic  level  1 had  been  activated  on  the  appropriate  keyboard. 

Throughout  each  flight,  the  Information  displayed  on  the  VSO  and 
HSD  was  dynamic  In  response  to  thrust,  bank,  and  pitch  Inputs.  However, 
the  flight  director  on  the  VSO  was  Inoperable  until  the  pilot  crossed 
waypoint  1.  Selection  of  COHN  or  NAV  on  the  appropriate  keyboard 
determined  whether  communication  or  navigation  status  was  displayed  on 
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the  left  MPD.  When  displayed,  the  navigation  status  on  the  left  MPD 
constantly  presented  new  information  such  as  aircraft  position,  time  and 
distance  to  the  next  waypoint,  etc.  Also,  the  communication  format  display 
presented  the  status  of  the  communication  radios.  The  pilot's  flying 
task  was  to  maintain  ground  speed  and  altitude  during  70-80  miles  of 
flight  and  keep  the  flight  director  symbol  centered  on  the  Vertical 
Situation  Display.  (See  Appendix  D for  flight  information.)  The  pilot's 
keyboard  task  was  to  complete  two  communication  changes  and  two  navigation 
updates.  These  tasks  were  felt  to  be  analogous  to  the  pilot's  flying 
a single  seat  fighter  aircraft. 

Prior  to  each  flight,  the  pilot  was  given  a Flight  Plan  (AF  Form  70) 
specifying  radio  frequencies.  Fifteen  UHF  radio  frequencies  were  each 
identified  by  a letter.  Instructions  to  change  a new  frequency  were 
given  orally,  using  control ler-to-pi lot  radio  ten;iinology.  By  identifying 
the  new  frequency  by  letter,  errors  due  to  forgetting  or  misunderstanding 
a four-digit  sequence  were  eliminated.  The  random  assignment  of  fre- 
quencies from  the  list  of  fifteen  prevented  the  pilot  from  anticipating 
or  memorizing  the  new  frequency  prior  to  task  assignment.  The  waypoint 
coordinates  to  be  entered  were  also  listed  on  the  AF  Form  70.  Since 
their  length,  and  the  uncertainty  of  whether  the  next  task  was  COMM  or 
NAV  tended  to  preclude  memorization,  extra  waypoints  were  not  included. 
Instructions  to  enter  or  "update"  a waypoint  were  given  by  a controller, 
using  standard  terminology  and  by  identifying  the  waypoint  with  a single 
number.  When  the  pilot  updated  the  waypoint,  he  entered  both  the 
waypoint  number  and  the  latitude/longitude  coordinates.  The  experimenter 
did  not  initiate  these  tasks  while  the  pilot  was  banking  or  before 
waypoint  1. 

The  appropriate  keyboards  were  inoperative  until  activated  by  the 
experimenter.  Activation  of  the  following  switches  was  required  for  a 
navigation  update:  KAV,  NAV  COMP,  WAfPT,  and  the  appropriate  digits. 

If  an  incorrect  switch  was  pushed  In  levels  1,  2,  or  3,  legends  unrelated 
to  the  requested  task  appeared  on  the  keyboard.  In  order  to  complete 
the  required  task,  the  pilot  had  to  correct  the  mistake.  To  accomplish 
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this,  the  pilot  pushed  the  CLEAR  switch  once  to  return  the  keyboard  to 
the  previous  level  and  then  made  the  proper  selection.  Once  the  pilot 
reached  the  fourth  logic  level  step,  a pre-entry  readout  of  each 
selected  digit  was  available  on  the  navigation  MPD  format.  If  an 
erroneous  digit  was  detected,  the  pilot  pushed  the  clear  switch.  This 
action  erased  all  the  digits  making  reselection  of  each  and  every  digit 
necessary.  In  the  opinion  of  the  experimenters,  this  was  not  the  optimum 
error  correction  method  but  was  retained  for  the  experiment  due  to  time 
restrictions  and  for  the  purpose  of  gathering  data  on  its  use.  When  the 
pilot  pushed  the  ENTER  button,  the  computer  interpreted  the  digits  selected 
and  determined  their  accuracy.  If  the  pilot  had  pushed  an  incorrect 
digit,  the  error  message,  "INCORRECT  DIGIT  ENTRY,  RE-ENTER,"  was 
presented  at  the  top  of  the  display  and  the  keyboard  returned  to  the 
third  level.  In  order  to  complete  the  task,  the  pilot  had  to  push 
WAYPT  again  to  activate  the  digits  and  repeat  the  entry.  Once  the 
correct  information  was  entered,  the  keyboards  returned  to  SENSORS  at 
logic  level  1 and  the  format  for  navigation  status  was  displayed 
on  the  left  MPD. 


For  a communication  change,  activation  of  the  following  switches 
was  required:  COMM,  UHF,  and  UHF  POWER,  A/N,  and  appropriate  digits. 

Both  UHF  and  UHF  POWER  switches  were  selected  at  step  2.  However,  during 
the  second  communication  change  of  each  flight,  the  UHF  power  remained 
active  so  that  the  required  switch  sequence  became;  COMM,  UHF,  A/N,  and 
digits.  The  MPD/ keyboard  changes  and  related  procedures  of  a communication 
change  were  similar  to  that  of  a navigation  update.  These  procedures 
differed,  however,  if  a pilot  entered  an  incorrect  frequency  that  was 
still  within  the  normal  UHF  frequency  range.  In  this  case,  the  keyboard 
returned  to  the  third  level  but  no  error  message  was  presented  on  the 
MPD.  The  pilot  was  then  notified  by  the  experimenter  to  redo  the  task. 


iv 


Concerning  the  normal /failure  status  of  the  configuration,  each 
flight  was  initialized  in  a normal  mode.  A change  of  the  normal/failure 
status  did  not  occur  during  task  events.  When  the  experimenter  changed 
the  configuration  to  a degraded  mode,  the  master  caution  light  located 
to  the  left  of  the  VSD  flashed  orange.  When  the  pilot  acknowledged  the 
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failed  state  by  pushing  the  master  caution  light,  the  flashing  stopped. 
Concurrently,  a primary  keyboard  became  inoperable  and  the  logic  levels 
that  were  on  the  keyboard  were  presented  on  the  right  console  keyboard. 

When  a failure  occurred,  the  display  on  the  right  MPD  was  replaced  with 
the  failure  message.  This  format  specified  which  keyboard  was  failed  and 
the  logic  levels  that  were  operable  on  the  right  console,  backup 
keyboard.  When  the  experimenter  returned  the  cockpit  to  a normal  mode, 
the  master  caution  light  flashed  green,  the  primary  keyboard  became 
operable,  the  right  console  keyboard  became  inoperable,  and  the  information 
displayed  on  the  right  MPD  indicated  that  normal  operation  was  reinstated. 

In  order  to  determine  the  effect  of  keyboard  operation  on  a pilot's 
ability  to  fly  the  aircraft,  flight  task  parameters  were  sampled  twenty 
times  per  second.  Further  data  concerning  keyboard  operation  was  obtained 
by  recording  the  time  required  to  complete  a task  event  and  the  number 
of  switch  hits  that  occurred  during  the  task  event.  For  purposes  of 
statistical  analysis,  a task  event  was  defined  as  follows:  (1)  the  pilot 
was  given  a request  for  a communication  change  or  navigation  update  by 
the  experimenter.  (Note  that  contrary  to  usual  flight  procedures,  the 
pilot  had  to  wait  for  a request  from  the  experimenter  prior  to  making  a 
navigation  update.  This  procedural  change  was  explained  to  the  pilot. 

The  pilot  was  required  to  respond  to  each  command  and  coniplete  the 
keyboard  operation  as  soon  as  possible.)  (2)  The  experimenter  pushed 
an  event  marker  switch  concurrent  with  the  pilot's  acknowledgement  of 
the  instructions.  Since  pilot  acknowledgement  tends  to  be  nearly 
automatic,  this  increases  the  probability  that  time  for  mental  processing 
and  decision  was  included  in  the  task  event  time.  (3)  If  the  keyboard 
data  was  entered,  but  was  incorrect  (i.e.,  wrong  frequency  or  waypoint 
coordinate  entered),  then  the  pilot  was  required  to  redo  the  necessary 
procedure  until  it  was  successfully  completed.  Time  to  complete  the  task 
and  number  of  switch  hits  were  measured  for  each  event  from  the  time 
the  pilot  responded  to  the  event  command  until  he  completed  the  sequence 
correctly.  An  upper  bound  of  four  minutes  was  operationally  defined  by 
the  experimenter  as  the  maximum  time  allowable  for  a keyboard  operation. 
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e.  Debriefing 

Following  the  completion  of  the  data  flights,  each  pilot  completed 
a form  concerning  his  background  flying  experience.  The  pilots  also 
filled  out  a questionnai»*e  designed  to  elicit  subjective  evaluations 
concerning  each  of  the  four  logic  level  arrangements,  the  location  of 
the  control  stick,  location  of  the  keyboards,  and  display  formats 
(Appendix  E). 

5.  PERFORMANCE  MEASURES  AND  DATA  ANALYSIS 

The  following  flight  parameters  were  recorded  twenty  times  per 
second  on  magnetic  tape; 

Ground  speed  (knots) 

Altitude  (feet) 

Flight  director  deviation  from  null  (arbitrary  units) 

Appropriate  summary  statistics  (average  error  (AE);  average  absolute 
error  (AAE);  root-mean-square  error  (RMS);  standard  deviation  (SD)  [see 
Appendix  F for  formulae])  were  computed  on  these  flight  parameters  for 
the  time  period  specified  by  the  event  and  for  the  immediate  fifteen 
seconds  prior  to  the  event.  The  fifteen  second  pre-event  time  was 
designated  as  baseline  performance.  Summary  statistics  for  baseline 
performance  for  each  parameter  were  subtracted  from  the  corresponding 
values  recorded  during  the  event  in  order  to  measure  only  the  effect  of 
the  keyboard  tasks  on  the  pilot's  performance.  This  difference  score 
quantified  the  level  of  the  flying  task  performance  decrement  expected 
due  to  keyboard  task  performance.  Keyboard  task  performance  was 
evaluated  by  measuring  the  time  required  for  the  task  and  the  number  of 
switch  hits.  Since  switch  hits  were  not  the  same  for  COMM  and  NAV  tasks, 
a Figure  of  Merit  (FOM)  was  computed  by  dividing  the  actual  number  of 
switch  hits  by  the  number  required  to  accomplish  the  task  without  error. 
For  an  example  computation,  see  Appendix  F.  An  error  free  task  would 
produce  a FOH  of  1.0.  As  errors  increased,  the  FOM  would  increase. 
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The  data  were  initially  analyzed  by  multivariate  analysis  of 
variance  (MANOVA)  using  the  BMD  12V  statistical  program  available  on  the 
CDC  6600  computer.  In  those  cases  where  the  MANOVA  revealed  significant 
effects,  subsequent  analyses  were  conducted  by  stepwise  discriminant 
function  analyses  (BMD07M)  in  order  to  determine  which  of  the  dependent 
variables  were  most  sensitive  to  changes  in  independent  variables.  The 
eight  dependent  variables  which  were  selected  for  these  analyses  are 
indicated  in  Table  1 by  an  asterisk. 

TABLE  .1 

SUMMARY  STATISTICS  FOR  EACH  DEPENDENT  VARIABLE 


The  following  is  a list  of  the  summary  statistics  calculated  for 
each  of  the  six  performance  variables  recorded  during  the  tasks. 


1 

* 2 
*$  3 

4 

5 

* 6 
*$  7 
8 
9 

10 
11 
12 
13 
* 14 
*$15 
16 
*$17 
*$18 


Dependent  Variable  Summary  Statistic 

Altitude  (feet)  AE 

AAE 

RMS 

SD 

Ground  speed  (knots)  AE 

AAE 

RMS 

SD 

Cross  track  error  (arbitrary  units)  AE 

AAE 

RMS 

SD 

Bank  error  (arbitrary  units)  AE 

AAE 

RMS 

SD 

Keyboard  operation  time  (seconds) 

Switch  hits  error  (figure  of  merit) 


* Variables  analyzed  by  multivariate  analysis  of 
variance  and  discriminant  function  analyses. 


$ Variables  used  in  the  analysis  of  work  discussed  in 
paragraph  I I 1-5  and  Appendix  6. 


I 

I 

p 


31 


AFFDL-TR-77-9 


In  the  first  phase  of  the  data  analysis,  communication  changes  were 
examined  separately  from  navigation  updates,  since  it  was  felt  that  the 
navigation  updates  were  more  demanding  and,  hence,  would  serve  as  a 
better  indication  of  any  treatment  effects.  In  addition,  since  it  was 
felt  that  the  keyboard  operations  completed  during  failed  modes  were 
more  demanding  than  those  completed  during  normal  modes,  analyses  of 
the  failed  mode  keyboard  operations  were  conducted  separately  from 
normal.  This  four-fold  categorization  resulted  in  communication  changes 
failed  mode,  navigation  updates  - failed  mode,  communication  changes  - 
normal  mode,  and  navigation  updates  - normal  mode.  In  order  to  analyze 
these  data,  four  separate  MANOVAs  were  run. 

In  regards  to  the  debriefing  questionnaire,  data  obtained  was 
compiled  to  be  presented  in  tabular  form  and  appropriate  summary 
statistics  were  calculated.  The  biographical  data  obtained  from  the 
flight  experience  questionnaire  was  also  evaluated  with  descriptive 
statistics  to  obtain  an  overall  view  of  the  characteristics  of  the 
pilot  sample. 

A work  analysis,  described  in  Appendix  6 was  conducted  to  assess 
the  total  effect  measured  by  changes  in  all  of  the  dependent  variables. 
The  output  of  this  analysis  is  a nondimensional  number  that  is  related 
to  the  percentage  of  work. 
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SECTION  IV 
RESULTS 

The  results  of  the  statistical  analyses  conducted  on  the  objective 
performance  measures  are  presented  for  each  of  the  following  areas  of 
investigation: 

a.  MFK  hardware  type 

b.  Logic  level  arrangement 

c.  Control  stick  location 

Within  each  area,  the  results  of  the  keyboard  operations  completed  during 
failed  conditions  are  presented  first,  followed  by  the  results  for  tasks 
completed  during  normal  conditions.  In  each  case,  the  results  for 
navigation  updates  are  discussed  separately  from  that  for  the  communication 
changes. 

1 . MFK  HARDWARE  TYPE 

a.  Navigation  Updates  Completed  Under  Failed  Conditions 

The  results  of  the  MANOVA  of  the  navigation  tasks  completed  under 
failed  conditions  revealed  a significant  hardware  main  effect  (F  » 3.22, 

^ » 8,  17,  £ < .05).  A stepwise  discriminant  analysis  indicated  that 
pilot  performance  was  better  when  the  projection  switches  were  used  and 
that  the  bank  AAE  was  the  dependent  variable  most  sensitive  to  differences 
between  the  two  hardware  types  (F  ■ 4.23,  ^ ■ 1,  142,  £ < ,05). 

However,  this  difference  must  be  viewed  considering  a hardware  type  by 
logic  level  arrangement  Interaction  (F  ■ 2.34,  ^ ■ 24,  49.91,  £ < .01). 
This  interaction  indicates  that  the  optimal  keyboard  type  was  determined 
by  which  of  the  four  logic  level  arrang«»ents  was  in  use.  The  logic 
level  arrangement  variable  as  a factor  in  this  interaction  is  presented 
in  more  detail  in  Section  IV-2a.  A subsequent  stepwise  discriminant 
analysis  indicated  that  keyboard  operation  tine  was  the  dependent 
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variable  most  sensitive  to  the  variables  in  the  logic  level  by  hardware 
interaction  (F  = 3.82,  ^ = 7,  136,  £ < .01).  The  keyboard  operation 
time  for  pilots  to  complete  navigation  updates  during  failed  conditions 
is  shown  in  Figure  22  for  each  logic  level  arrangement  as  a function  of 
MFK  type.  As  illustrated  in  the  figure,  keyboard  operation  time  on  the 
right  console  MFK  was  faster  with  the  projection  switches  than  with  the 
plasma  planel  for  logic  level  arrangements  A,  C,  and  D.  In  the  case 
of  logic  level  arrangement  A,  which  involved  digit  entry  on  the  right 
console  MFK  during  failed  conditions,  the  difference  bet>/een  the 
projection  switches  and  the  plasma  panel  was  significant  < .01).  In 
other  words,  keyboard  operation  time  was  worse  v(hen  the  pilot  had  to 
input  digits  on  the  right  console  plasma  panel  keyboard.  Results  for 
logic  level  arrangement  B,  however,  indicated  that  while  not  statistically 
significant,  keyboard  operation  time  was  slightly  slower  with  the 
projection  switches  on  the  right  console  than  with  the  plasma  panel 
XB  < .25).  Inspection  of  Figure  8 shows  that  for  failure  conditions, 
logic  level  arrangement  B involved  the  cse  of  the  opposite  hardware  types 


Figuro  22.  Mean  Keyboard  Operation  Time  Required  for  Completion 
of  Navigation  Updates  During  Failed  Conditions  with 
earh  Logic  Level  Arrangement  as  a Function  of  MFK 
Hardware  Type 
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for  three  of  the  four  steps.  With  this  in  mind,  it  can  be  stated  that, 
for  logic  level  arrangement  B in  the  failed  condition,  the  increased 
time  required  for  "projection  switches"  was  accounted  for  by  the  fact 
that  the  plasma  panel  located  on  the  front  instrument  panel  was  used  for 
all  but  the  first  entry! 

For  these  navigation  tasks,  with  failed  panels,  the  analysis  of  work 
showed  that  the  work  required  with  a plasma  panel  was  nearly  twice 
standard  (work  factor  = 1.99)  while  the  increase  when  using  projection 
switches  was  only  1.63  times  the  standard.  According  to  the  classification 
scheme  used,  the  logic  level  arrangements  involving  either  a failure  of 
the  front  panel  MFK  or  the  digit/mode  panel  were  Identified  by  the  type 
of  hardware  mounted  on  the  right  console.  For  example,  in  the  case  of 
logic  level  arrangement  B in  which  the  plasma  pa-’s!  MFK  is  mounted  on  the 
right  console,  the  classification  scheme  designates  this  configuration 
as  operation  of  the  plasma  panel  MFK  under  failed  conditions.  Inspection 
of  Figure  8 indicates,  however,  that  this  arrangement  required  greater 
use  (three  of  the  four  logic  levels)  of  the  front  panel  projection  switch 
MFK  than  the  right  console  plasma  panel  MFK.  Therefore,  the  results  of 
the  analyses  conducted  using  this  classification  system  do  not  really 
reflect  operation  of  the  right  console  panel  since  most  of  the  switch 
actions  were  completed  on  the  front  panel.  This  discrepancy  occurred  only 
with  logic  level  arrangement  B.  If  the  classification  system  is  changed 
such  that  each  configuration  is  identified  according  to  which  MFK 
hardware  was  used  to  complete  the  majority  of  switch  hits,  the  plasma 
panel  work  required  is  2.12  times  standard  and  projection  switches  only 
1.5  times  the  standard  work.  These  numbers  are  not  qualitatively  different 
from  1.99  and  1.63,  but  the  apparent  anomaly  of  logic  level  arrangement  B 
is  eliminated.  When  the  work  required  for  each  logic  level  is  calculated 
for  the  two  hardware  types,  it  becomes  obvious  that,  for  this  task,  the 
projection  switches  required  a lower  level  of  work  (Figure  23). 

b.  Communication  Changes  Completed  Under  Failed  Conditions 

Analysis  of  the  coomunication  changes  completed  during  failed 
conditions  also  indicated  that  pilot  performance  significantly  differed 
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depending  on  the  MFK  hardware  type  used  (F  = 4.69,  ^ = 8,  17,  £ < .01) 
A stepwise  discriminant  analysis  identified  keyboard  operation  time 
(F  = 16.94,  ^ = 1,  142,  £ < .01;  Figure  24)  and  altitude  AAE  (£  = 4.43 
^ = 1,  141,  £ < .05,  Figure  25)  as  the  performance  measures  most 
sensitive  to  MFK  hardware  type  differences. 
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♦This  logic  level  arrangement  is  classified  according  to 
which  panel  was  front  mounted  and  used  for  all  but  the 
first  switch  hit  during  failed  conditions. 

Figure  23,  Work  Factor  for  Each  MFK  Hardware  Type  as  a Function 
of  Logic  Level  Arrangement. 
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Figure  24.  Mean  Keyboard  Operation  Tine  During  Failed  Conditions 
with  Each  MFK  Hardware  Type 
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Ptuma  Pan«i  MFK  Prolaotion  Switeh  MFK 
MFK  HARDWARE  TYPE 

Figure  25.  Delta  Altitude  AAE  for  Each  MFK  Hardware  Type  During 

Communication  Changes  Completed  Under  Failed  Conditions 

Mean  time  for  projection  switch  operation  was  22.0  seconds, 
compared  with  30.6  seconds  for  the  plasma  panel.  Altitude  AAE  was 
24.7  feet  for  projection  switches  and  80.2  feet  for  the  plasma  panel. 
Both  measures  showed  better  performance  with  the  projection  switches. 

Using  the  work  factors  derived  as  described  In  Appendix  G,  the 
plasma  panel  required  0.71  times  the  standard  while  the  projection 
switches  required  only  0.55  times  the  standard. 

c.  Navigation  Updates  Completed  Under  Normal  Conditions 

The  analysis  of  the  navigation  updates  completed  during  normal 
conditions  revealed  that  performance  was  significantly  different 
depending  upon  which  type  of  MFK  hardware  was  used  (£  » 4.08,  ^ “ 8,  17, 
£ < .01).  The  results  of  a stepwise  discriminant  analysis  of  this  data 
showed  that  bank  AAE  (F  ■ 7.25,  ^ ■ 1,  142,  £ < .01)  and  bank 
RMS  (F  = 3.43,  ^ ■ 1,  141,  £ < .10)  were  the  dependent  variables  most 
sensitive  to  the  type  of  hardware  used.  Inspection  of  Figure  26 
Illustrating  bank  AAE  and  RMS  for  each  keyboard  type  Indicates  that  the 
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PiMma  PaiMi  MFK  Projaeilon  Switch  MFK 
MFK  HARDWARE  TYFE 

Figure  26.  Delta  Bank  AAE  and  RMS  for  Each  MFK  Hardware  Type  During 
Navigation  Updates  Completed  Under  Normal  Conditions 

pilots  had  greater  difficulty  maintaining  bank  while  completing  navigation 
updates  when  the  projection  switches  were  on  the  front  Instrument  panel 
as  compared  to  performance  with  the  plasma  panel  on  the  front  Instrument 
panel.  In  other  words,  In  contrast  with  both  failed  condition  tasks,  pilot 
performance  was  better  using  the  plasma  panel  when  completing  navigation 
updates  under  normal  conditions. 

While  the  work  factors  confirm  better  performance  with  the  plasma 
panel  (1.49  vs.  1.54),  the  difference  was  small. 

d.  Communication  Changes  Completed  Under  Normal  Conditions 

The  MANOVA  of  communication  changes  completed  under  normal  conditions 
revealed  a significant  MFK  hardware  type  by  control  stick  location 
Interaction  (F  ■ 3,21,  dj[  ■ 8,  17,  £ < .05).  A stepwise  discriminant 
analysis  Identified  keyboard  operation  time  as  the  dependent  variable 
most  sensitive  to  these  factors  (F  « 3.25,  ^ ■ 3,  140,  £ < .05;  Figure  27). 
The  plasma  panel/center  stick  configuration  resulted  In  the  worst 
performance  by  the  pilots  (23.6  seconds).  When  compared  to  this  worst 
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MFK  HAROWME  TYRE 

Figure  27.  Mean  Keyboard  Operation  Time  with  Each  Control  Stick 
Location  as  a Function  of  MFK  Hardware  Type 

case  configuration,  keyboard  operation  time  was  significantly  faster 
with  the  projection  switches,  for  both  center  stick  (17.8  seconds; 

F = 8.87,  df  = 1,  140,  £ < .01)  and  side  stick  (19.2  seconds;  F = 5.09, 
df  = 1,  140,  £ < .05)  location.  No  other  significant  differences  were 
found  with  other  comparisons. 

With  a center  stick,  the  work  factors  verified  the  superior 
performance  of  the  projection  switches  versus  the  plasma  panel  (0.45  vs. 
0.63).  The  finding  that  there  was  only  a small  difference  between  the 
hardware  types  when  a side  stick  was  used  (0.57  and  0.51),  tends  to 
support  the  above  results  that  the  difference  between  these  two  are  not 
significant. 

e.  NFK  Hardware  Type:  Summary 

In  brief,  analyses  of  both  coBiounlcatlon  changes  and  navigation 
updates  during  failed  conditions  indicated  that  pilot  performance  was 
better  on  the  right  console  NFK  with  the  projection  switches  than  with 
the  plasma  panel.  Specifically,  keyboard  time  and  altitude  AAE  measures 
were  better  for  coanunication  changes  completed  during  failed  conditions 
In  that  time  was  less  and  errors  smaller  with  the  projection  switches 
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in  comparison  to  the  plasma  panel.  Keyboard  operation  time  was  also 
generally  faster  with  the  projection  switches  than  with  the  plasma  panel 
hardware  for  the  navigation  updates  completed  during  failed  conditions. 

The  results  further  suggested  that  performance  was  degraded  whenever  the 
digits  were  entered  on  the  plasma  panel  keyboard. 

The  analysis  of  communication  changes  completed  under  normal 
conditions  using  the  front  instrument  panel  MFK  also  indicated  that 
pilot  performance  was  better  in  terms  of  keyboard  operation  time  with 
the  projection  switches  than  with  the  plasma  panel.  In  contrast,  the 
analysis  of  navigation  updates  completed  under  normal  conditions  using 
the  front  instrument  panel  MFK  showed  that  performance  was  better  in 
terms  of  keyboard  operation  time  and  maintaining  bank  with  the  plasma 
panel  than  with  the  projection  switches. 

2.  LOGIC  LEVEL  ARRANGEMENT 

a.  Navigation  Updates  Completed  Under  Failed  Conditions 

Initial  analysis  revealed  a significant  main  effect  of  logic  level 
arrangement  (F  « 2.18,  ■ 24,  49.91,  £ < .05).  The  stepwise  discriminant 

analysis  indicated  that  the  time  variable  was  most  sensitive  to  the  main 
effect  (F  ■ 2.96,  ^ ■ 3,  140,  £ < .05).  Performance  was  significantly 
worse  with  logic  level  arrangement  A (78.3  seconds)  than  that  for 
arrangements  C (59.1  seconds)  and  D (59.9  seconds;  £<  .05).  Additionally, 
although  not  statistically  significant,  performance  was  somewhat  worse 
with  logic  level  arrangement  B (74.5  seconds)  than  that  for  arrangements  C 
and  0 (£  < .10).  These  performance  differences,  however,  should  be 
examined  In  light  of  the  significant  Interaction  discussed  below. 

As  mentioned  earlier,  a significant  Interaction  of  logic  level 
arrangement  and  NFK  hardware  type  was  found  In  the  MANOVA  conducted  on 
the  navigation  updates  completed  during  failed  c.^ndltlons  (£  < .01). 

An  Interaction  of  these  factors  Indicates  that  ^e  optimal  logic  level 
arrangement  was  determined  by  adiether  the  pilot  used  the  plasma  panel  or 
the  projection  switches.  The  results  of  a stepwise  discriminant  function 
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analysis  indicated  that  keyboard  operation  time  was  the  dependent  variabl 
most  sensitive  to  these  factors  (see  Table  2 for  F matrix).  The  mean 
time  required  for  completion  of  a navigation  update  with  each  MFK 
hardware  type  is  illustrated  in  Figure  28  as  a function  of  logic  level 
arrangement.  As  can  be  seen  in  the  figure,  performance  was  worse  when 
pilots  were  using  logic  level  arrangement  A on  the  plasma  panel. 
Examination  of  Table  2 indicates  that  this  decrement  in  performance  was 
significant  at  the  .01  level  for  all  configurations  except  for  the 
projection  switches/logic  level  arrangement  B configuration,  in  which 
case  there  was  only  a marginal  statistical  difference  (£  < .25).  The 
discriminant  function  analysis  also  indicated  that  keyboard  operation 
was  significantly  worse  using  logic  level  arrangement  B,  with  the 
projection  switches  located  on  the  right  console  than  performance  with 
arrangements  A (£  < .05),  C (£  < .01),  and  D (£  < .05)  with  the  same 
MFK  hardware.  Further  inspection  of  Figures  28  and  8 shows  that 
keyboard  operation  time  was  greater  when  the  digits  were  entered  on  the 
plasma  panel  and  least  when  the  separate,  dedicated  Oigit/Mode  Panel 
was  used  for  digit  entry  (arrangements  C and  0). 
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Figure  28.  Man  Keyboard  Operatieii  Time  with  Each  MFK  Hardware 
Type  as  a Function  of  Logic  Level  Arrangement 


AFFOL-TR-77-9 


AFFDL-TR-77-9 


The  work  analysis  supported  the  superiority  of  logic  level 
arrangements  C (work  factor  = 1.55)  and  D (1.66)  over  A (2.11)  and 
B (1.92).  When  the  hardware  effect  was  taken  into  consideration,  the 
smallest  work  factor  using  plasma  panel  (with  a work  factor  of  1.79)  was 
larger  than  the  largest  work  factor  using  the  projection  switches  (1.68). 
Logic  level  arrangement  C was  superior,  regardless  of  whether  plasma 
panel  (1.70)  or  projection  switches  (1.31)  were  used. 

b.  Communication  Changes  Completed  Under  Failed  Conditions 

Significant  performance  difference  among  logic  level  arrangements 
were  found  in  the  MANOVA  of  the  communication  changes  completed  during 
failed  conditions  (F  = 1.92,  = 24,  49.91,  p < .05)  The  results  of 

the  stepwise  discriminant  analysis  identified  altitude  AAE  as  the 
dependent  variable  most  sensitive  to  these  differences  (F  = 2.51,  ^ = 3, 
140,  £<  .10;  Figure  29).  Although  the  difference  was  not  statistically 
significant,  the  results  are  in  agreement  with  those  obtained  during 
navigation  tasks  under  failed  conditions.  Pilot  performance  was 
significantly  worse  with  logic  level  arrangement  B (86.7  ft)  than  for 
arrangements  C (24.5  ft)  and  D (34.1  ft;  £<  .05).  Furthermore,  although 
not  statistically  significant,  performance  with  arrangement  A (64.8  ft) 
was  worse  than  C and  0 at  the  .25  level,  thereby,  indicating  the  same 
trend.  Namely,  that  logic  level  arrangements  C and  D Involved  the  use 
of  the  dedicated  Oigit/Mode  Panel  on  the  left  console  for  entering  digits 
whereas,  logic  level  arrangements  A and  B required  the  use  of  the  MFK 
for  digit  entry  (Figure  8). 

The  work  equation  confirmed  these  results.  Arrangements  C and  D 
were  essentially  the  same  (work  factor  ■ 0.60).  Arrangement  B was  the 
most  difficult  (0.72).  Arrangement  A had  an  intermediate  work  factor 
0.63. 


c.  MFK  Operation  Under  Normal  Conditions 

MANOVAs  of  communication  changes  and  navigation  updates  completed 
during  normal  conditions  indicated  there  ^ere  no  signi  Meant  differences 
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Figure  29.  Delta  Altitude  AAE  for  Each  Logic  Level  Arrangement  During 
Communication  Changes  Completed  Under  Failed  Conditions 


among  the  four  logic  level  arrangements.  In  other  words,  during  normal 
conditions,  pilot  performance  was  essentially  the  same  regardless  of  the 
particular  logic  level  arrangement  (A,  B,  C,  or  D)  used  to  complete  the 
keyboard  operations. 

d.  Logic  Level  Arrangement:  Sunnary 

To  summarize,  the  analyses  of  keyboard  operations  completed  under 
normal  conditions  did  not  reveal  performance  differences  among  logic 
level  arrangements,  whereas  those  for  failed  conditions  .did.  Performance 
was  generally  worse  when  the  digits  were  entered  on  the  plasma  punel 
and  best  when  the  separate,  dedicated  Dlgit/Mode  Panel  was  used  for 
digit  entry.  Two  cases  are  noteworthy.  In  the  first  case,  for  the 
analysis  of  navigation  updates  completed  on  the  right  console  MFK  under 
failed  conditions,  keyboard  operation  was  slower  with  logic  level 
arrangement  A using  the  plasma  panel  on  the  right  console  than  any  other 
keyboard/logic  level  arrangement  configuration.  In  the  second  case, 
keyboard  operation  was  slower  with  logic  level  arrangement  B using  the 
projection  switches  on  the  right  console  than  that  for  arrangements  A,  C, 
and  0 with  the  same  keyboard  on  the  right  console.  In  both  of  these 
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cases,  the  logic  level  arrangements  were  such  that  the  digits  were 
entered  on  the  plasma  panel  type  MFK.  In  a separate  analysis  of 
communication  changes  completed  under  failed  conditions,  similar  results 
were  found  indicating  that  performance  was  worse  in  terms  of  maintaining 
altitude  with  logic  level  arrangements  A and  B than  for  C and  D. 

3.  CONTROL  STICK  LOCATION 

a.  MFK  Operation  Under  Failed  Conditions 

Analyses  of  navigation  updates  and  communication  changes  completed 
under  failed  conditions  did  not  indicate  any  significant  differences 
due  to  control  stick  location.  In  other  words,  pilot  performance  during 
failed  conditions  was  essentially  the  same  regardless  of  where  the  control 
stick  was  located. 

b.  Navigation  Updates  Completed  Under  Normal  Conditions 

No  significant  differences  due  to  control  stick  location  were  found 
in  the  analysis  of  navigation  updates  completed  under  normal  conditions. 

c.  Communication  Changes  Completed  Under  Normal  Conditions 

As  mentioned  in  Section  IV-l.d,  a significant  interaction  of  control 
stick  location  and  MFK  hardware  type  was  found  in  the  MANOVA  conducted 
on  the  communication  changes  completed  under  normal  conditions  (£  < .05). 
An  interaction  of  these  factors  indicates  that  the  optimal  control  stick 
location  during  normal  communication  changes  was  determined  by  whether 
the  pilot  used  the  plasma  panel  or  projection  switches  on  the  front  panel. 
A stepwise  discriminant  function  analysis  identified  time  as  the  dependent 
variable  most  sensitive  to  these  factors  (|i  < .05).  Referring  to 
Figure  27,  there  was  a significant  difference  in  time  between  the  center 
stick/plasma  panel  configuration  and  both  configurations  using  the 
projection  switches  (center  stick/projection  switches,  £<  .OU  side 
stick/projection  switches,  £<  .05). 


AFFDL-TR-77-9 


d.  Control  Stick  Location;  Summary 

Analyses  of  communication  and  navigation  keyboard  operations  under 
failed  conditions  and  analysis  of  navigation  updates  completed  under 
normal  conditions  failed  to  reveal  significant  differences  due  to  control 
stick  location.  The  analysis  of  communication  changes  completed  under 
normal  conditions,  however.  Indicated  that  it  took  more  time  to  complete 
tasks  on  the  plasma  panel  MFK  than  on  the  projection  switch  type  MFK 
regardless  of  which  control  stick  location  was  used.  Performance  was 
especially  degraded  with  the  center  stick/plasma  panel  configuration. 
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SECTION  V 
DISCUSSION 

In  this  section,  the  results  reported  in  Section  IV  are  discussed. 

For  simplicity,  the  same  presentation  order  of  topics  and  findings  is 
used.  Findings  are  interpreted  and  explanations  suggested  for; 

a.  MFK  hardware  type 

b.  Logic  level  arrangement 

c.  Control  stick  location 

When  applicable,  the  subjective  responses  of  the  participating  pilots 
are  referenced. 

1 .  MFK  HARDWARE  TYPE 

A projection  switch  type  MFK  was  compared  with  a plasma  panel  MFK. 
Both  MFKs  were  examined  under  failed  conditions  (right  console)  and  under 
normal  conditions  (front  panel)  for  communication  changes,  as  well  as 
navigation  updates. 

In  addition  to  the  basic  physical  difference  between  the  plasma  and 
projection  hardware,  there  were  design  differences  between  the  two  hard- 
ware implementations.  In  particular,  problems  were  noted  with  the  plasma 
panel  MFK  due  to  the  following  design  features: 

1.  The  white  tape  lines  radiating  between  legends  and  switches  on 
the  plasma  panel  were  not  parallel  because  the  switches  took  more  space 
than  the  legends. 

2.  The  switches  on  the  plasma  panel  were  smaller  and  mounted  closer 
together  than  the  switches  used  on  the  projection  switch  HPK. 

3.  The  dimensions  of  the  plasma  panel  surface  caused  the  two 
columns  of  switches  to  be  nearly  a foot  apart.  This  physical  separation 
of  the  two  columns  of  switches  made  the  operation  of  the  switches  on 
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the  right  hand  side  of  the  panel  more  difficult,  especially  when 
mounted  on  the  right  console. 

4.  Parallax  problems  resulted  when  the  plasma  panel  was  on  the 
right  console  due  to  the  fact  that  the  switches  on  the  plasma  panel  were 
not  flush-mounted,  and  the  panel  was  not  mounted  at  the  optimal  viewing 
angle. 

It  is  important  to  note  that  these  ancillary  design  differences 
could,  by  themselves,  account  for  any  “hardware"  differences  found  in 
the  analyses.  Care  must  be  taken  not  to  generalize  any  specific  findings 
for  these  MFK  configurations  to  other  plasma  panels  or  projection  switches 
at  other  cockpit  locations. 

a.  MFK  Operation  Under  Failed  Conditions 

Under  failed  conditions,  there  is  no  doubt  about  the  results 
with  respect  to  hardware  type:  the  projection  switch  panel  used  in  this 
study  was  better  than  the  plasma  panel  used.  Pilot  performance  was 
better  for  projection  switches,  regardless  of  task  difficulty.  (The 
navigation  task  with  19  switch  hits  was  considered  to  be  more  difficult 
that  the  communication  task  with  8 switch  hits.)  Both  tasks  took  longer 
when  the  plasma  panel  was  used.  Aircraft  control  parameters  (bank  and 
altitude  AAE)  showed  that  pilot  performance  on  the  flying  task  deteriorated 
when  the  plasma  panel  was  used.  Seven  of  the  nine  pilots  stated  that, 
during  failed  conditions  the  projection  switches  were  much  easier  to 
operate.  Some  pilots  commented  that  the  legends  on  the  lighted  pro- 
jection switches  made  selection  of  the  appropriate  switch  much  faster. 
Concerning  the  plasma  panel,  the  pilots  commented  that  operation  was 
difficult  due  to  the  parallax  problems,  radiating  lines,  small  switch 
size,  and  switch  arrangement. 

b.  NFK  Operations  Under  Normal  Conditions 

The  analyses  of  the  results  from  the  MFK  operations  on  the  front 
panel  are  conflicting.  Subjective  data  indicated  that  the  pilots  felt 
that  the  projection  switches  were  easier  to  operate  (six  of  the  nine 
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said  "much  easier").  The  performance  analyses  supported  this  conclusion 
for  the  communication  changes.  However,  unlike  the  findings  for  the 
normal  communication  changes  (projection  switches  better),  the  performance 
for  navigation  updates  completed  under  normal  conditions  showed  the 
plasma  panel  MFK  to  be  better. 

Examination  of  the  design  differences  between  the  two  MFKs  suggests 
possible  explanations  for  these  findings.  First,  it  should  be  noted 
that  since  normal  operation  involved  the  use  of  the  front  panel  location, 
parallax  problems  associated  with  the  mounting  angle  on  the  right  console 
were  reduced  for  both  the  communication  changes  and  navigation  updates. 
Secondly,  the  separated  columns  on  the  plasma  panel  were  more  of  a 
detrimental  factor  during  communication  changes  compared  to  navigation 
updates.  For  example,  the  communication  changes  executed  on  the  plasma 
panel  required  as  many  as  three-fourths  of  the  total  switch  hits  on  the 
right  side  of  the  MFK.  Since  most  pilots  operated  the  switches  with  the 
left  hand  while  flying  with  the  right,  reaching  the  switches  on  the  right 
side  of  the  panel  was  difficult,  especially  when  a center  stick  was  used. 
This  problem,  combined  with  the  fact  that  the  MFK  operation  during 
communication  changes  involved  alternating  between  the  left  and  right 
sides  of  the  separated  columns,  tended  to  make  plasma  panel  operation 
more  difficult.  Thus,  even  though  parallax  was  not  a problem  during 
completion  of  communication  changes  on  the  plasma  panel  during  normal 
conditions,  three  of  the  four  design  deficiencies  (radiating  lines, 
small  switch  size,  and  separated  columns)  still  affected  performance. 

It  is  suggested  that,  for  this  reason,  the  results  of  the  normal  com- 
munication changes  were  in  agreement  with  those  obtained  under  the 
failed  conditions  (i.e.,  projection  switch  better).  Regarding  the 
navigation  updates,  however,  the  negative  effects  of  both  parallax  and 
separated  columns  were  reduced.  For  updates,  less  than  one-third  of 
the  not'tnal  navigation  switch  hits  required  the  use  of  switches  on  the 
right  side  of  the  plasma  panel.  By  using  nainly  the  switches  on  the  left 
side  of  the  panel,  the  effect  of  widely  separated  columns  was  reduced. 

It  is  possible  that  the  reduction  of  these  effects,  In  this  case, 
accounts  for  the  fact  that  plasma  panel  operation  was  found  easier  than 
projection  switch  operation.  At  the  very  least,  these  findings  cast 
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some  doubt  on  any  attempt  to  generalize  conclusions  about  the  hardware 
types  and  indicate  that  further  investigation  is  necessary. 


Another  possible  explanation  for  the  better  performance  with  the 
plasma  panel  during  normal  navigation  updates  is  that  the  more  optimally 
designed  projection  switch  logic  encouraged  the  subjects  to  work  faster 
and  devote  more  attention  to  keyboard  operation.  This  apparently  caused 
the  pilot  to  ignore  the  flying  task,  as  shown  by  the  performance  decrement 
in  maintaining  bank,  during  failed  navigation  updates.  However,  considering 
the  difficulty  of  navigation  updates,  this  could  result  in  the  pilots 
making  more  errors  in  the  keyboard  task.  In  fact,  pilots  did  accomplish 
the  navigation  tasks  faster  using  the  projection  switches  (52.5  seconds 
vs.  56.1  seconds)  and  had  a higher  error  rate  using  the  projection 
switches  (1.12  vs.  1.10).  The  trends  shown  by  these  results,  while  not 
significant,  do  tend  to  support  this  explanation. 


The  data  also  suggests  that  the  pilots  devoted  more  attention  to 
keyboard  operation  during  normal  communication  changes  completed  on  the 
projection  switches.  Since  fewer  switch  hits  were  required  for  the 
communication  changes  compared  to  the  navigation  updates,  it  appears 
that  the  pilots  who  devoted  full  attention  to  projection  switch  operation 
were  able  to  complete  the  task  before  flying  task  errors  developed,  and 
were  able  to  complete  the  keyboard  task  with  fewer  errors.  Both  attitude 
and  bank  errors  were  less  with  the  projection  switches  than  the  plasma 
panel.  In  addition,  operation  with  the  projection  switches  MFK  was 
faster  (18.51  vs.  22.14  sec)  and  had  a lower  error  rate  (1.03  vs. 

1.08)  compared  to  the  plasma  panel.  The  trends  shown  by  these  results 
do  tend  to  support  this  explanation.  Conclusive  evidence  is  not 
available,  but  could  be  obtained  by  further  experiments  with  optimized 
design  of  the  candidate  panels. 

2.  LOGIC  LEVEL  ARRANGEMENT 

Four  logic  level  arrangements  were  evaluated.  They  Involved  changes 
In  the  location  of  switches  used  for  steps  in  the  operating  sequence 
(see  Figure  8). 
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a.  MFK  Operations  Under  Failed  Conditions 

During  operations  under  failed  conditions  (increased  task 
difficulty),  the  advantages  of  a separate  Digit/Mode  Keyboard  for  logic 
levels  1 and/or  4,  whether  due  to  the  dedicated  switches  or  optimized 
number  arrangement,  is  apparent  as  shown  by  the  significantly  better 
pilot  performance  under  logic  level  arrangements  C and  D.  Contrary  to 
these  findings,  the  subjective  data  indicated  that  the  pilots  preferred 
logic  level  arrangement  B.  This  was  apparently  due  to  the  fact  that 
only  one  switch  hit  was  required  on  the  right  console  MFK.  All  other 
switch  hits  in  logic  level  arrangement  B were  on  the  front  panel  MFK. 
When  considering  only  the  logic  level  arrangements  where  the  front  panel 
was  failed,  the  pilots  preferred  arrangement  C.  The  pilots  unanimously 
agreed  that  operation  of  the  MFK  was  very  inefficient  with  arrangement  A 
which  required  the  use  of  the  MFK  for  digit  entry.  This  latter  finding, 
in  conjunction  with  the  results  of  the  performance  analyses,  suggests 
that  logic  level  arrangements  which  use  locations  that  are  hard  to  see 
and  reach,  such  as  the  vertical  mounting  on  the  right  console,  should 
be  avoided. 

b.  MFK  Operations  Under  Normal  Conditions 

A most  noteworthy  result  concerning  logic  level  arrangement 
was  the  failure  to  find  any  significant  differences  during  normal 
operation.  This  implies  that,  as  long  as  the  panels  are  readable  and 
reachable  with  no  control  stick  interference,  the  distribution  of  logic 
levels  among  panels  does  not  affect  pilot  performance.  Performance 
trends  showed  that  logic  level  arrangements  C and  0 which  involved  digit 
entry  on  the  separate,  dedicated  Digit/Mode  Panel  tended  to  be  better 
than  arrangements  A and  B which  required  the  use  of  the  MFK  for  digit 
entry.  Subjective  data  Indicated  that  for  normal  operation,  pilots 
preferred  arrangement  0 which  Involved  operation  of  the  front  panel 
MFK  for  the  first  three  logic  level  steps  and  the  Olgit/Node  Panel  for 
the  fourth. 
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3.  CONTROL  STICK  LOCATION 

The  use  of  both  a center  and  side  control  stick  location  was 
examined  during  the  experiment.  It  was  not  the  intent  of  the  study  to 
evaluate  differences  in  stick  location  but  rather  the  effects  of  stick 
location  on  MFK  operation. 

a.  MFK  Operations  Under  Failed  Conditions 

Concerning  operation  of  the  right  console  MFK  with  the  side 
control  stick,  eight  out  of  nine  pilots  responded  that  interference 
problems  occurred  with  this  configuration.  The  pilots  also  indicated 
that  slightly  less  interference  resulted  when  the  center  stick  location 
was  used  with  the  right  console  MFK.  Some  commented  that  this  configuration 
made  it  possible  to  fly  with  the  left  hand  and  operate  the  keyboard  with  the 
right.  Note,  however,  that  contrary  to  the  pilots'  responses  on  the 
questionnaire,  performance  analyses  did  not  indicate  any  significant 
findings  in  respect  to  the  control  stick  location  with  the  right  console  MFK. 

b.  MFK  Operations  Under  Normal  Conditions 

The  results  of  the  analyses  of  the  front  panel  MFK  operation 
during  normal  conditions  revealed  effects  due  to  control  stick  location 
for  communication  changes  but  not  for  navigation  updates.  The  analyses 
indicated  that  when  the  right  side  switches  of  the  plasma  panel  were 
obstructed  by  the  center  stick  during  the  normal  communication  changes, 
operation  was  degraded.  It  is  suggested  that  this  control  stick  effect 
occurred  in  the  communication  changes  as  opposed  to  the  navigation  updates 
because  the  UHF  radio  changes  involved  more  operation  of  the  switches  on  the 
right  side  including  the  "A/N"  button  in  the  lower  right  corner  of  the 
plasma  panel.  Even  though  the  performance  analyses  showed  interference 
between  the  center  stick  and  the  plasma  panel,  only  two  pilots  indicated 
on  the  questionnaire  that  the  center  stick  intcrferred  with  the  front 
panel.  Eight  out  of  nine  pilots  did  state,  however,  that  the  side  stick 
location  aided  the  operation  of  the  front  panel  MFK  and  some  further 
commented  that  this  configuration  allowed  full  unobstructed  view  of  the 
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front  panel.  This  suggests  that  the  center  stick  location  obstructed 
the  view  of  the  front  panel. 

From  these  analyses,  it  can  be  concluded  that,  so  long  as  MFK^ 
operation  is  not  directly  inhibited  by  the  location  of  the  control 
stick,  performance  is  not  affected.  In  other  words,  by  proper  location 
of  the  MFK,  its  operation  is  compatible  with  the  control  stick  in  either 
the  center  or  right  side  position. 
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SECTION  VI 
CONCLUSIONS 

As  a result  of  this  evaluation  on  the  use  of  multifunction  keyboards 
(MFKs)  in  single-seat  Air  Force  cockpits,  the  following  conclusions  can 
be  made: 

(1)  Progressing  through  the  four  levels  of  system  control  was 
effective.  The  overall  time  required  for  pilots  to  complete 
communication  changes  and  navigation  updates,  during  simulated 
flight  was  around  20  seconds  and  55  seconds,  respectively. 

(2)  Operations  other  than  digit  entry  should  be  consolidated  on  a 
single  keyboard. 

(3)  The  digit  entry  should  be  completed  on  a separate  dedicated 
panel . 

(4)  The  location  of  the  control  stick  (center  or  side)  does  not 
affect  performance  as  long  as  MFK  operation  is  not  physically 
inhibited  by  the  stick. 

(5)  Plasma  panel  type  keyboards  should  be  designed  such  that  the 
legends  on  the  panel  are  directly  adjacent  and  in  line  with  the 
corresponding  switches. 

(6)  The  viewing  angle  of  keyboards  is  an  important  factor  in 
keyboard  operation.  This  is  especially  critical  when  the 
switches  are  not  flush-mounted  witli  the  panel. 

{7}  The  switches  should  be  located  on  the  keyboard  such  that 

hand  movements  alt'^rnating  between  the  left  and  right  sides  of 
the  panel  are  minimized. 
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APPENDIX  A 
COCKPIT  DISPLAYS 

Four  electro-optical  displays  were  used  in  the  present  study  to 
provide  information  for  utilization  by  the  pilot  (Figure  9).  The 
following  describes  each  display  in  detail: 

a.  Vertical  Situation  Display  (VSD) 

A nine  inch  diagonal  color  monitor  presented  flight  symbology 
to  the  pilot.  The  symbology  consisted  of  (1)  a white  horizon  line 
delineating  the  boundary  between  a blue  sky  and  brown  earth  background, 
(2)  a white  pitch  ladder  with  5 degree  increments  for  the  first  i 30 
degrees  from  the  horizon  line  and  ten  degrees  thereafter,  (3)  black  roll 
indexes  every  ten  degrees  (+  60  degrees)  with  a white  roll  index  marker. 
(4)  a flight  director  symbol  (active  in  bank  only)  in  orange,  and  (5)  a 
fixed  black  aircraft  symbol.  Altitude,  indicated  airspeed,  heading, 
vertical  velocity,  and  acceleration  (g*s)  were  presented  digitally  in 
white  on  black  background.  In  addition,  trend  information  for  the 
airspeed  and  altitude  parameters  was  provided  by  white  thermometer  type 
bars  which  were  placed  above  the  respective  digital  readouts.  (See 
Figure  16  for  a representation.) 

b.  Horizontal  Situation  Display  (HSO) 

A nine  inch  diagonal  monochrome  monitor  presented  simplified 
navigation  information  in  a heading  up  format.  The  symbology  consisted 
of  (1)  a triangular  aircraft  symbol,  (2)  a symbolic  flight  path  between 
mission  waypoints,  and  (3)  digital  readouts  of  ground  speed,  heading,  and 
distance  to  the  next  waypoint.  All  symbology  w&s  green  on  a black  back- 
ground. In  addition,  the  Une  representing  the  flight  path  became  Jagged 
or  stairstepped  when  the  aircraft's  heading  did  not  parallel  the  flight 
path.  (See  Figure  17.) 
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c.  Multipurpose  Displays  (MPDs) 

Two  five  inch  monochrome  monitors  were  used.  The  one  mounted 
on  the  left  side  of  the  cockpit  provided  either  communication  data  or 
navigation  data  (Figures  18  and  19).  Engine  instrumentation  and  keyboard 
failure  status  information  was  displayed  on  the  right  MPD  (Figures  ?0 
and  21).  This  MPD  had  sixteen  peripheral  push  button  switches.  However, 
for  the  present  study,  these  switches  remained  inoperable. 
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APPENDIX  B 

EXPERIMENTER'S  CONSOLE  AND  SIMULATOR  FACILITIES 

1.  EXPERIMENTER'S  CONSOLE 

The  console's  four',  six  inch  diagonal,  monochrome  CRT  displays, 
prov’-led  the  experimenter  with  the  capacity  of  monitoring  the  simulator 
displays  (VSD,  HSD,  and  2 MPDs).  The  experimenter  was  also  able  to 
initiate  tasks  and  control  the  normal /failure  status  of  the  configurations. 
In  order  to  minimize  experimenter  workload,  cockpit  reconfiguration 
was  automated  as  much  as  possible  and  incorrect  waypoint  and  frequency 
digits  were  detected  by  the  computer. 

2.  SIMULATOR  FACILITIES 

The  simulator  consisted  of  four  interconnected  facilities  as  shown 
in  Figure  Bl.  A functional  description  of  each  system  element  is 
provided  below. 

a.  POP  11/45 

Configuration  Control  - used  to  set  up  the  cockpit  controls/ 
displays  configuration  prior  to  each  flight. 

Display  Assembly  - generated  Image  listings  to  be  further 
processed  by  the  Ramtek  raster  symbol  generator.  Data  from  the  simulation 
models  was  used  for  the  VSD  and  NPD  formats. 

Map  Driver  - provided  output  control  of  nap  data  to  the 
Ramtek  symbol  generator.  The  Image  lists  of  the  map  were  done  by 
the  DEC  10. 

Keyboard  Logic  - processed  incoming  switch  data  and  determined 
the  display  state  of  all  the  keyboards. 

night  Control  Sampling  and  Scaling  - buffered  and  scaled 
flight  control  data  to  be  used  by  the  DEC  10  simulation  models. 
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b.  DEC  10 

Simulatio!^  Models  - provided  all  necessary  aircraft  parameters 
to  the  11/45  to  be  used  in  display  processing. 

Map  Assembly  - generated  a display  list  of  symbolic  waypoint 
map  information  to  be  processed  by  the  Ramtek  symbol  generator. 

Data  Recording  - recorded  cockpit  display  parameter  data  on 
magnetic  tape  at  a 20  per  second  iteration  rate. 

Data  Reduction  - an  off-line  program  reduced  the  raw  real-time 
recorded  data  into  meaningful  data  that  can  be  analyzed. 

c . Ramtek 

Display  Generation  - Processed  image  lists  to  display  VSD,  HSD, 
and  MPDs  on  525  line  raster  monitors. 

d.  Cockpit 

Keyboard  Input/Output  - provided  a switch  Image  buffer  of  all 
cockpit  switch  states  to  be  sampled  by  the  11/45.  Also  decoded  keyboard 
display  data  being  sent  from  the  11/45. 

Flight  Control  - Digitized  analog  stick,  rudder,  and  thrust 
control  inputs  and  buffered  the  resultant  data  for  transmission  to 
the  11/45. 
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APPENDIX  C 

DAILY  TEST  SCHEDULES 

The  following  daily  test  schedules  (Tables  Cl  through  C3)  indicate 
the  time  and  activity  to  train,  test,  and  debrief  two  pilots  during  six 
consecutive  days  of  the  experiment.  Alternate  schedules  were  also 
available  for  training  a pilot  and  testing  a pilot  in  one  day,  testing  a 
pilot  and  debriefing  a pilot  in  one  day,  etc.  Times  for  controls/displays 
familiarization,  training  flights,  test  flights,  and  simulator  recon- 
figurations are  indicated  in  the  schedules  provided. 
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TABLE  01 

TEST  SCHEDULE  FOit  TRAINING  TWO  PILOTS 

Day  1 


Time 

Activity 

1245  - 1315 

Introduction  to  Simulation,  Facility,  and  Purpose 

of  Study/Pilot  A 

1315  - 1345 

Familiarization  with  simulator/Pilot  A 

1345  - 1355 

Practice  Trial  1/Pilot  A 

1355  - 1405 

Software  change 

1405  - 1415 

Practice  Trial  2/Pilot  A 

1415  - 1425 

Software  change 

1425  - 1435 

Practice  Trial  3/Pilot  A 

1435  - 1445 

Software  change 

1445  - 1455 

Practice  Trial  4/Pilot  A 

1455  - 1500 

Software  change/control  stick  change 

1500  - 1515 

Introduction  to  Simulation/Pilot  B 

1515  - 1545 

Familiarization  with  Simulator/Pilot  B 

1545  - 1555 

Practice  Trial  1/Pilot  B 

1555  - 1605 

Software  change 

1605  - 1615 

Practice  Trial  2/Pilot  B 

1615  > 1625 

Software  change 

1625  - 1635 

Practice  Trial  3/Pilot  B 

1635  - 1645 

Software  change 

1645  - 1655 

Practice  Trial  4/Pilot  B 
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TABLE  C2 

TEST  SCHEDULE  FOR  TESTING  TWO  PILOTS 

Days  2-5 


Time 

Activity 

1300  - 1310 

Trial  1 /Pi lot  A 

1310  - 1320 

Software  change 

1320  - 1330 

Trial  2/Pilot  A 

1330  - 1340 

Software  change 

1340  - 1350 

Trial  3/Pilot  A 

1350  - 1400 

Software  change 

1400  - 1410 

Trial  4/Pilot  A 

1410  - 1420 

Software  change 

1420  - 1430 

Trial  5/Pilot  A 

1430  - 1450 

Software  change/control  stick  change 

1450  - 1500 

Trial  1/Pilot  B 

1500  - 1510 

Software  change 

1510  - 1520 

Trial  2/Pilot  B 

1520  - 1530 

Software  change 

1530  - 1540 

Trial  3/Pilot  B 

1540  - 1550 

Software  change 

1550  - 1600 

Trial  4/Pilot  6 

1600  - 1610 

Software  change 

1610  - 1620 

Trial  5/Pilot  B 

Computer  time  per  day  ■ 200  minutes  for  10  test  runs. 
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TABLE  C3 

TEST  SCHEDULE  FOR  TESTING  AND  DEBRIEFING  TWO  PILOTS 

Day  6 


Time 

Activity 

1300  - 1310 

Trial  1/Pilot  A 

1310  - 1320 

Software  change 

1320  - 1330 

Trial  2/Pilot  A 

1330  - 1340 

Software  change 

1340  - 1350 

Trial  3/Pilot  A 

1350  - 1400 

Software  change 

1400  - 1410 

Trial  4/Pilot  A 

1410  - 1430 

Software  change/control  stick  change 

Debrief  Pilot  A 

1430  - 1440 

Trial  1/Pilot  B 

1440  - 1450 

Software  change 

1450  - 1500 

Trial  2/Pilot  B 

1500  - 1510 

Software  change 

1510  - 1520 

Trial  3/Pilot  B 

1520  - 1530 

Software  change 

1530  - 1540 

Trial  4/Pilot  B 

1540  - 1640 

Debrief  Pilot  B 

Computer*  Time  ■ 160  minutes  for  eight  test  runs. 
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APPENDIX  D 
FLIGHT  INFORMATION 

A total  of  six  missions,  four  for  test  flights  and  two  for  training 
flights,  were  used  in  the  experiment.  Initial  conditions  for  all  the 
missions  are  specified  in  Table  Dl.  The  programmed  track  for  each 
mission  was  70  nm  with  a turn  of  approximately  90  degrees  at  waypoint  2. 

At  420  knots  ground  speed,  the  total  flight  segment  lasted  for  ten  minutes. 
The  flight,  however,  was  terminated  at  the  completion  of  the  fourth  task, 
even  if  the  pilot  did  not  reach  the  end  of  the  track.  Figure  01  shows 
a sample  AF  Form  70  which  was  given  to  the  pilot  prior  to  each  flight. 

The  forms  provided  information  pertaining  to  radio  frequencies  and 
waypoint  coordinates  for  use  during  the  secondary  tasks. 
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TABLE  D1 


INITIAL  CONDITIONS  FOR  FLIGHT  MISSIONS 


Training  Flight  1. 

Altitude 
Ground  Speed 
Heading 
Location 


Training  Flight  2. 

Altitude 
Ground  Speed 
Heading 
Location 


Test  Flight  1. 

Altitude 
Ground  Speed 
Heading 
Location 


Je%t  Flight  2. 

Altitude 
Ground  Speed 
Heading 
Location 


Test  Flight  3. 

Altitude 
Ground  Speed 
Heading 
Location 


20,000' 
420  knots 
165“ 

144825  N 
1065045  E 


20,000* 
420  knots 
288* 

141104  N 
1074040  E 


20,000* 

420  knots 

150730  N 
1083815  E 


20,000’ 
420  knots 
188* 

154720  H 
1081459  E 


20,000* 
420  knots 
178* 

161330  N 
1080540  E 


Test  Flight  4. 

Altitude  • 20,000* 

Ground  Speed  - 420  knots 

Heading  - 270* 

Location  • 153435  U 

1083605  E 
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APPENDIX  E 

PILOT  QUESTIONNAIRES 

The  pilot  data  questionnaire  form  concerning  background  flying 
experience  was  completed  by  each  pilot  subject  sometime  during  the 
experiment.  Following  the  simulation,  the  pilots*  filled  out  a Likert- 
type  rating  questionnaire  designed  to  elicit  subjective  evaluations 
concerning  each  of  the  four  logic  level  arrangements,  the  location  of 
the  control  stick,  MFK  type  and  location,  and  display  formats.  The  copy 
of  the  debriefing  questionnaire  Included  In  this  appendix  also  provides 
the  pilots'  responses. 
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PILOT  DATA 

Mcirne : 

Rank:  ■ 

Present  Duty  Assiftim.ent: 

Organization  , 

Syi.ibol : 

Total  Active  Duty: 

Date  Pilot  Rating  Obtained 
Total  Fljring  Hours 
Hours  in  A/C  hy  Tyne: 
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PILOT  DEBRIEFING  QUESTIONNAIRE 

The  purpose  of  this  questionnaire  is  to  find  out  what  you  think 
about  this  cockpit.  It  is  not  a test  and  you  may  take  as  long  as  you 
need. 


Your  candid  opinions  will  help  in  the  evaluation  of  this  cockpit. 
Please  answer  all  the  questions  by  indicating  the  response  which  most 
nearly  describes  your  feelings.  When  a question  reminds  you  of  some 
particular  comment,  or  when  you  want  to  explain  your  answer,  feel  free 
to  write  your  feelings  in  the  extra  space  provided. 

The  questionnaire  is  divided  into  three  sections  representing 
the  specific  areas  of  interest  at  this  time.  The  first  section  deals 
primarily  with  each  of  the  four  logic  level  arrangements  between  the 
primary  and  backup  keyboards.  The  second  section  is  concerned  with 
the  location  of  the  control  stick  and  keyboo'^d  type,  while  the  third 
section  deals  with  the  formats  used  on  other  displays. 
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KEYBOARD  LOGIC  LEVEL  ARRANGEMENT 

You  may  have  noticed  that  each  new  plane  you  fly  has  more  and  more 
control  panels,  and  that  you  need  to  make  like  a pretzel  to  reach  some 
of  them. 

Operations  specialists  tell  us  we  need  more  sophisticated  sub-systems. 

Avionics  experts  tell  us  each  system  needs  a separate  control  panel. 

Pilots  tell  us  that  there  are  already  too  many  switches,  knobs,  dials, 
and  warning  lights. 

One  solution  is  to  design  a Multifunction  Keyboard  (MFK)  that  will 
allow  the  pilot  to  control  several  sub-systems  with  a single  panel. 

Catch  22:  if  this  single  panel  fails  (and  you  know  who  will  be  flying 
the  plane  in  what  kind  of  weather  when  it  does)  all  those  sub-systems 
(COMM,  NAV,  WEAPONS,  IFF,  etc.)  fail  with  it.  Engineers  call  this  a 
degraded  mode.  A solution  to  this  problem  is  to  put  a backup  keyboard 
in  the  cockpit. 

The  systems  being  studied  here  have  four  step  procedures  which  we 
call  logic  levels.  When  a panel  fails,  the  log^c  levels  are  automatically 
redistributed  to  other  panels.  Figure  A shows  Tour  different  distributions 
of  the  four  logic  levels  which  are  used  for  "normal"  operation  in  the 
present  experiment.  Figure  B shows  alternate  distributions  of  the  four 
logic  levels  for  "failure"  or  "degrr. 'ed"  operation. 

Answer  the  following  questions  in  this  manner:  for  each  question, 
put  the  number  of  the  response  corresponding  to  your  feelings  under  the 
letter  representing  the  appropriate  logic  level  arrangement.  Please 
identify  any  comments  you  make  with  the  appropriate  logic  level  arrange- 
ment letter. 
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NORMAL  CONDITION 


FP  “ Front  Panel 
LC  » Left  Console 
RC  » Right  Console 


Figure  A.  Keyboard  operation  for  each  logic 
level  arrangement  during  normal 
conditions. 
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FAILED  CONDITION 


F.P  = Front  Panel 
LC  = Left  Console 
RC  = Right  Console 


B l-FP  S-PP 


0 »ic»«c*»w4u: 


Fioure  B.  Keyboard  operation 
^ logic  level  arrangement  during 

failed  conditions. 
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I-l.  During  normal  operation,  progressing  through  the  four  logic  levels 
to  enter  data  on  the  MFK  was 


A B C D 


1. 

Very  inefficient 

2. 

Moderately  inefficient 

2/9 

1/9 

3/9 

- 

3. 

Moderately  efficient 

4/9 

7/9 

3/9 

5/9 

4. 

Very  efficient 

3/9 

1/9 

3/9 

4/9 

Comments: 

- For  example,  changing  radio  frequency  is  easier  with  MFK  method 
because  presently  with  standard  radios,  you  have  to  usually  check 
the  frequency  after  inserted.  With  the  MFK  you  can  punch  in 
numbers  by  "touch"  much  as  you  would  type  by  touch. 

- It  is  easier  and  more  efficient  if  the  same  keyboard  can  be  used 
for  all  levels. 

- I found  the  left  console  the  most  convenient  to  operate,  particu- 
larly when  putting  in  the  waypoints.  Its  operation  required  the 
smallest  physical  movement  frcxn  the  manual  position. 

- Since  the  majority  of  comm  tasks  involve  changing  frequencies, 
isn't  it  possible  to  make  that  a 3 step  or  2 step  operation  simply 
by  using  another  key? 

- Having  steps  1-4  on  the  same  panel  is  a plus.  Using  the  left 
console  would  be  the  optimal  for  me. 
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1-2.  Considering  the  overall  mission  workload  during  normal  conditions, 
operation  of  the  MFK 

A B C D 

1.  Interfered  greatly  with  completion 
of  other  mission  tasks 

2.  Interfered  slightly  with  completion 

of  other  mission  tasks  4/9  2/9  4/9  1/9 

3.  Did  not  interfere  with  completion 

of  other  mission  tasks  4/9  6/9  4/9  7/9 

4.  Aided  slightly  with  completion  of 

other  mission  tasks  1/9  1/9  1/9 

5.  Aided  greatly  with  completion  of 

other  mission  tasks  1/9 

Comments: 

- The  more  tasks  that  were  on  the  front  panel,  the  more  deviations 
I could  pick  up  with  my  peripheral  vision  on  the  ADI. 

I-3A.  During  failure  operation,  progressing  through  four  logic  levels 
to  enter  data  on  the  MFK  was 


A B 

C 

D 

1. 

Very  inefficient 

9/9 

3/9 

6/9 

2. 

Moderately  inefficient 

4/9 

6/9 

3/9 

3. 

Moderately  efficient 

5/9 

4.  Very  efficient 

Comments: 

- Using  the  right  hand  panel  was  difficult.  Turning  the  head  to 
see  the  panel  and  then  back  to  the  ADI  can  cause  vertigo. 

- On  the  plasma  panel,  it  was  very  difficult  to  determine  which 
button  corresponded  to  which  function  and  made  the  process  of 
completing  a task  much  slower  and  more  difficult. 
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I-3B.  Suppose  we  put  the  backup  keyboard  at  the  right  MPD  location 

instead  of  on  the  right  hand  console.  During  failure  operation, 
progressing  through  the  four  logic  levels  to  enter  data  on  the 


MFK  would  be 

A 

B 

C 

D 

1. 

Very  inefficient 

1/9 

2/9 

2/9 

2. 

Moderately  inefficient 

5/9 

2/9 

4/9 

5/9 

3. 

Moderately  efficient 

2/9 

7/9 

3/9 

2/9 

4. 

Very  efficient 

1/9 

Comments: 

- Having  to  reach  with  the  left  hand  to  the  right  hand  side  of 
the  cockpit  can  cause  problems.  Use  the  left  MPD  instead. 

- How  about  left  MPD  for  convene! ence?  Also  depends  on  stick 
location. 

- It  is  easier  and  more  efficient  if  the  same  keyboard  can  be 
used  for  all  levels. 

- Right  hand  would  already  be  occupied  controlling  aircraft  with 
center  and  side  stick.  This  would  necessitate  reaching  cross 
cockpit  with  the  left  arm  to  operate  MPD. 

- Would  probably  be  better  with  side  stick,  otherwise,  I would 
prefer  backup  on  left  MPD  location. 
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1-4,  Considering  the  overall  mission  workload  during  failure 
conditions,  operation  of  the  MFK 

A B C D 

1.  Interfered  greatly  with  completion 

of  other  mission  tasks  9/9  3/9  7/9 

2.  Interfered  slightly  with  completion 

of  other  mission  tasks  7/9  6/9  2/9 

3.  Did  not  interfere  with  completion 

of  other  mission  tasks  2/9 

4.  Aided  slightly  with  completion  of 
other  mission  tasks 

5.  Aided  greatly  with  completion  of 
other  mission  tasks 

Comments : 

- Too  much  head  shifting  around  in  Logic  Level  C. 

- Plasma  panel  bad. 

- During  a failure  mode,  why  not  switch  aJi  4 steps  to 
another  MFK. 

- Using  the  right  hand  panel  was  difficult.  Turning  the  head 
to  see  the  panel  and  then  back  to  the  ADI  can  cause  vertigo. 
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For  the  following  questions,  fill  in  the  circle  which  most  nearly 
described  your  feelings  about  the  object  of  the  question.  Space  is 
provided  for  any  additional  comments  you  might  have. 

1-5.  Concerning  the  arrangement  of  the  four  logic  levels,  keyboard 
operation  is 

Much  easier  on  2 MFKs  compared  to  1 MFK 
1/9  Moderately  easier  on  2 MFKs  compared  to  1 MFK 
3/9  Equally  easy  on  2 MFKs  and  1 MFK 
4/9  Moderately  easier  on  1 MFK  compared  to  2 MFKs 
1/9  Much  easier  on  1 MFK  compared  to  2 MFKs 
Comments: 

- It  all  depends  on  the  location  of  the  keyboards. 

- Depends  more  on  arrangement  and  location  than  number. 

1-6.  The  MFK  should  contain 

More  keys  and  more  logic  levels 
2/9  More  keys  and  fewer  logic  levels 
4/9  The  number  of  keys  and  logic  levels  as  it  now  has 
1/9  Fewer  keys  and  more  logic  levels 
2/9  Fewer  keys  and  fewer  logic  levels 
Comments: 

- Since  the  majority  of  comm  tasks  involve  changing  frequencies, 
isn't  it  possible  to  make  that  a 3 step  or  2 step  operation 
simply  by  using  another  key? 

- There  Is  obvoiusly  a minimum  number  possible. 

- I don't  want  to  have  to  do  mental  calculations  (which  logic 
level  am  I in)  when  things  are  going  wrong  in  the  weather. 


f - 
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1-7.  The  MPD  pre-entry  readout  was 

5/9  Very  useful  In  detecting  data  input  errors  from  the  MFK 
3/9  Slightly  useful  in  detecting  data  input  errors  from  the  MFK 
1/9  Not  useful  in  detecting  data  input  errors  from  the  MFK 
Comments: 

- Absolutely  essential. 

- On  navigation  entries,  it's  a necessity. 

- It  might  be  of  better  use  if  readout  number  were  bigger  and/or 
brighter  than  other  numbers. 

- It  was  too  cluttered  with  information.  Printing  needs  to  be 
larger  and  more  prominent. 

1-8.  The  control  actions  required  to  correct  Incorrect  entries  in 
the  MFK  were 

3/9  Very  easy  to  perform 
2/9  Moderately  easy  to  perform 
2/9  Moderately  difficult  to  perform 
2/9  Very  difficult  to  perform 
Comments: 

- Difficult  especially  on  the  navigation  entry  (where  I made 
most  errors).  Especially  if  two  panels  were  involved,  I would 
forget  where  to  back  up  to  start  over. 

- Difficult  In  the  sense  that  It  required  repeating  the  task 
almost  from  the  beginning. 

- You  shouldn't  have  to  clear  the  entire  entry  because  you  get 
the  last  digit  wrong  after  entering  18  correctly. 

- Kept  forgetting  level  to  which  the  system  reverted  with  clear 
button. 
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- Very  easy  to  perform  but  frustrating  at  times  after  making  a 
mistake  on  the  last  number  of  a waypoint  and  then  having  to  go 
through  the  whole  series  of  tasks  and  number  again.  Maybe  a 
back  space  design  similar  to  the  clear  on  the  logic  step  could 
be  developed  so  that  if  a mistake  is  made,  you  could  back 
space  and  erase  a number  or  numbers  until  your  data  is  correct. 
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SECTION  II 

EFFECT  OF  CONTROL  STICK  LOCATION  AND 
TYPE  OF  KEYBOARD  ON  KEYBOARD  OPERATION 

Both  a center  and  a side  stick  location  were  used  in  the  present 
experiment.  The  purpose  of  examining  both  locations  is  to  determine  if 
stick  placement  interferes  with  the  operation  of  the  multifunction  key- 
boards. Another  factor  under  consideration  in  this  study  concerns  the 
relative  ease  of  keyboard  operation  in  respect  to  the  type  of  keyboard, 
location  of  the  keyboard,  and  characteristic  switch  size,  legend,  etc. 

The  following  section  specifically  addresses  these  considerations.  As 
before,  please  fill  in  the  appropriate  circle  which  describes  your 
feelings  about  each  configuration  and  note  any  comments  you  might  have. 
Figures  C-E  should  be  utilized  when  answering  questions  in  this  section. 

II-l.  The  placement  of  the  control  stick  in  a center  location 

Interfered  greatly  with  the  operation  of  the  front  panel 
keyboard 

2/9  Interfered  slightly  with  the  operation  of  the  front  panel 
keyboard 

7/9  Did  not  interfere  with  the  operation  of  the  front  panel 
keyboard 

Aided  slightly  with  the  operation  of  the  front  panel 
keyboard 

Aided  greatly  with  the  operation  of  the  front  panel 
keyboard 

Comments : 

- The  placement  of  the  control  stick  In  a center  location  did 
not  Interfere  with  the  operation  of  the  front  panel  keyboard. 
In  the  135,  the  column  Is  always  In  the  way  so  I didn't  notice 
anything  unusual.  I Just  worked  around  It. 
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- Much  easier  to  operate  front  panel  when'*projection  panel  is 
in  front,  plasma  panel  keys  too  close  to  operate  efficiently. 

- The  placement  of  the  control  stick  in  a center  location 
interfered  slightly  with  the  operation  of  the  front  panel 
keyboard  because  one  had  to  reach  over  the  top  of  the  stick 
to  push  the  "A/N"  button. 


Figure  C,  Side  end  center  locations  of  the  control  stick. 
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II-2.  The  placement  of  the  control  stick  in  a center  location 

2/9  Interfered  greatly  with  the  operation  of  the  right  hand 
console  keyboard 

4/9  Interfered  slightly  with  the  operation  of  the  right  hand 
console  keyboard 

Did  not  interfere  with  the  operation  of  the  right  hand 
console  keyboard 

1/9  Aided  slightly  with  the  operation  of  the  right  hand  console 
keyboard 

2/9  Aided  greatly  with  the  operation  of  the  right  hand  console 

Comments: 

- You  are  able  to  hold  a much  more  stable  flying  platform  with 
center  stick  when  reaching  across  with  left  arm  to  operate 
right  hand  panel. 

- The  placement  of  the  control  stick  in  a center  location  aided 
slightly  with  the  operation  of  the  right  hand  console  keyboard. 
I found  I could  hold  the  stick  with  my  left  hand  and  use  iny 
right  to  work  the  keyboard. 

- The  placement  of  the  control  stick  In  a center  locution  inter- 
fered greatly  with  the  operation  of  the  right  hand  console 
keyboard  because  reaching  for  the  buttons  was  awkward. 

- The  placement  of  the  control  stick  in  a center  location  aided 
greatly  with  the  operation  of  the  right  hand  console,  t flew 
with  iny  left  hand  and  punched  with  the  right.  However,  when 
doing  this  it*s  better  to  have  the  keyboard  words  to  the  left 
of  the  tMitton  so  my  hand  doesn't  cover  the  labels  when  punching 
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II-3.  The  placement  of  the  control  stick  in  a side  location 

7/9  Interfered  greatly  with  the  operation  of  the  right  hand 
console  keyboard 

1/9  Interfered  slightly  with  the  operation  of  the  right  hand 
console  keyboard 

1/9  Did  not  interfere  with  the  operation  of  the  right  hand 
console  keyboard 

Aided  slightly  with  the  operation  of  the  right  hand 
console  keyboard 

Aided  greatly  with  the  operation  of  the  right  hand 
console  keyboard 

Coa»nents : 

- Placement  of  the  control  stick  in  a side  location  inter- 
fered greatly  with  the  operaticn  of  the  right  hand  console 
keyboard  because  reaching  for  the  buttons  was  awkward. 

- Placement  of  the  control  stick  in  a side  stick  location 
made  it  hard  to  see  the  keyboard  and  maintain  control  of 
the  aircraft. 

- The  placement  of  the  control  stick  in  a side  location  inter- 
fered greatly  with  the  operation  of  the  right  hand  console 
keyboard.  Can’t  change  hands. 

M-4.  The  placement  of  the  control  stick  in  a side  location 

Interfered  greatly  w'U  . the  openatim  of  |he  front  p^^ 
keyboard. 

Interfered  slightly  with  the  operation  of  the  front  panel 
keyboard. 
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1/9  Did  not  Interfere  with  the  operation  of  the  front  panel 
keyboard 

7/9  Aided  slightly  with  the  operation  of  the  front  panel 
keyboard 

1/9  Aided  greatly  with  the  operation  of  the  front  panel 
keyboard 

Cominents: 

- Nothing  obstructed  my  view;  thus,  I could  think  about  my 
operations  as  sooii  as  the  task  was  given  and  I determined  the 
configuration. 

- Side  stick  with  front  panels  and.  right  MPO  instead  of  plasma 
panel  on  side  should  be  very  easy  to  operate.  The  pilot 
could  easily  monitor  position,  altitude,  attitude,  etc. 

While  completing  conmunication  or  navigation  tasks. 

- The  placement  of  the  control  stick  in  a side  location  did 
not  interfere  with  the  operation  of  the  front  panel  keyboard. 
The  reason  it  did  not  aid  me  is  because  with  a side  stick  I 
was  more  relaxed  and  leaned  towards  the  stick  more,  thus 
making  it  harder  to  reach  the  front  panel. 

' Configuration  with  side  stick  gives  you  full  unobstructed 
view  of  front  panel. 


U-5.  The  plasma  panel  was  located  either  on  the  front  panel  or  on  the 
right  hand  console.  Concerning  the  location  of  the  plasma  panel 

9/9  Operation  was  much  easier  when  the  plasma  panel  was  located 
on  Che  front  console 

Operation  was  slightly  easier  when  the  plasma  panel  was 
located  on  the  front  console 
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Operational  ease  of  the  plasma  panel  was  the  same  on  the 
front  console  and  right  hand  console 

Operation  was  slightly  easier  when  the  plasma  panel  was 
located  on  the  right  hand  console 

Operation  was  much  easier  when  the  plasma  panel  was 
located  on  the  right  hand  console 

Comments: 

- The  plasma  panel  was  almost  unusable  when  on  the  right  console 
and  marginal  on  the  cente*^  console  for  levels  1,  2,  and  3. 
"A/N"  input  was  impossible  on  right  side. 

- The  switches  not  being  aligned  with  the  readout  caused  visual 
tracking  (parallax)  problems. 

- You  could  tell  at  a glance  what  was  happening. 

- Even  though  the  front  console  location  was  better,  it  doesn't 
mean  I like  that  particular  system. 

- Easier  on  the  front  panel,  however,  keys  are  too  close  to  use 
efficiently  especially  in  turbulent  flying  conditions. 

Suggest  staggering  the  buttons. 

- There  was  still  a problem  aligning  the  plasma  panel  display 
with  Its  relative  button  on  the  front  panel,  but  not  nearly 
as  great  as  when  it  was  on  the  right  hand  console. 

• Parallax  biggest  problem.  Also  inefficient  use  of  space 
anywhere. 

- It  was  difficult  to  correspond  buttons  to  functions  when 
the  plasma  panel  was  on  the  side  panel. 

' Buttons  on  plasma  panel  were  too  small  for  operation  with 
gloves  on.  Much  parallax  In  lines  connecting  switches  and 
numbers. 
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II-6.  The  projection  switches  were  also  located  either  on  the  front 

panel  or  on  the  right  hand  console.  Concerning  the  location  of 

the  projection  switches 

5/9  Operation  was  much  easier  when  the  projection  switches 
were  located  on  the  front  console 

2/9  Operation  was  slightly  easier  when  the  projection  switches 
were  located  on  the  front  console 

1/9  Operational  ease  of  the  projection  switches  were  the  same 
on  the  front  console  and  right  hand  console 

1/9  Operation  was  slightly  easier  when  the  projection  switches 
were  located  on  the  right  hand  console 

Operation  was  much  easier  when  the  projection  switches 
were  located  on  the  right  hand  console 

Comments: 

- Ease  of  projection  switches  was  a combination  of  head  movement 
and  visibility.  Legibility  of  projection  switches  is  not 
great  and  having  them  out  front  is  a help. 

- The  lighted  switch  made  the  choice  process  much  faster  and  as 
a backup  would  be  easy  to  use. 

- Operational  ease  of  the  projection  switches  were  the  same  on 
the  front  console  and  right  hand  console.  It  was  much  easier 
than  the  plasma  panel  on  the  side  as  the  buttons  had  the 
numbers/functions  right  on  them. 

- Operation  of  projection  switches  panel  very  easy.  Buttons  are 
large  and  well  separated. 

• Operation  was  much  easier  when  the  projection  switches  were 
located  on  the  front  console.  But  easier  on  side  panel  than 
plasma  panel  on  side  panel. 
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II-7.  The  legend  on  the  plasma  panel  was 
Very  difficult  to  read 
2/9  Moderately  difficult  to  read 
3/9  Moderately  easy  to  read 
4/9  Very  easy  to  read 
Comments : 

- Except  the  parallax  made  it  bad  and  the  slanted  lines  necessary 
because  the  height  of  the  button  column  is  greater  than  the 
word  panel.  If  the  white  guide  lines  from  button  to  words 
were  parallel  and  horizontal,  it  would  be  much  better. 

- Legend  was  moderately  difficult  to  read,  but  difficult  to  track. 

- Rather  small  letters  and  buttons  to  push. 

II-8.  Selecting  the  appropriate  switch  on  the  plasma  panel  from  the 
available  legend  was 

6/8*  Very  difficult 

1/8  Moderately  difficult 

1/8  Moderately  easy  (*0ne  subject  did  not  indicate  a 

response.) 

Very  easy 
Comments : 

- Parallax  and  lack  of  consistent  associatiort  i.je. , top  switch  = 
top  function,  makes  plasma  panel  the  worst  of  the  three  and 
virtually  unusable  in  the  right  hand  console  position. 

- If  the  panel  was  set  up  properly  (i.e.,  tailored  logic  concept 
mentioned  by  experimenter),  I see  no  problem  with  operation. 

- Very  hard  to  figure  out  which  button  to  push.  Took  a lot  more 
concentration. 
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- Selecting  the  appropriate  switch  on  the  plasma  panel  from  the 
available  legend  was  very  difficult.  Only  with  plasma  panel 
on  right  side  panel.  Moderately  easy  when  on  the  front  panel. 

- Buttons  are  too  close  to  use  efficiently.  Lines  to  readout  from 
buttons  are  very  deceptive  if  not  looked  at  carefully  and 

from  head  on. 

II-9.  How  did  the  size  of  the  switches  and  their  proximity  to  each 
other  on  the  plasma  panel  affect  your  ability  to  operate  the 
keyboard  while  wearing  flyirg  gloves? 

4/9  Made  operation  very  difficult 

4/9  Made  operation  moderately  difficult 

1/9  Did  not  affect  operation 

Hade  operation  moderately  easy 

Made  operation  very  easy 

Comments : 

- Size  was  a little  small  and  they  were  too  close  together.  But 
the  left  row  was  too  far  away  from  the  right  row  to  go  back 
and  forth. 

> More  space  between  keys  would  be  helpful.  Also,  some  changes 

in  shape  may  be  useful  for  night;  i.e.,  raised  dots  or  depressions 
for  the  first  key  of  each  column. 

> Sizes  of  switches,  etc.  on  plasma  panel  made  operation  very 
difficult.  Only  with  plasma  panel  on  right  side  panel. 

Moderately  easy  when  on  the  front  panel. 

- Staggering  buttons  would  facilitate  ease  of  operation. 

II-IO.  Considering  your  previous  association  of  amber  with  emergencies, 
the  amber  color  of  the  plasma  panel  legends  was 
Very  confusing 
Moderately  confusing 

9/9  Not  confusing  at  all 
Comments; 

No  comments. 
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Figure  El,  Legends  on  plasma  panel  keyboard  for  each  logic 
level  step  during  a coRiminicatlon  task. 
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How  did  the  placement  of  the  digit  "1/N"  on  the  plasma  panel 
(second  key  on  left)  affect  your  operation  of  the  keyboard? 

3/9  Made  operation  very  difficult 

4/9  Made  operation  moderately  difficult 

1/9  Did  not  affect  operation 

1/9  Made  operation  moderately  easy 

Made  operation  very  easy 

Conments: 

- Should  have  been  placed  to  be  operated  by  top  switch.  Tape 
lines  were  of  little  help  since  they  ended  between  words  on 
the  panel  and  were  very  confusing, 

- I tended  to  associate  the  “1/N"  digit  with  the  first  key  on 
top. 

• The  placement  of  the  digit  “1/N**  on  the  plasma  panel  made 
operation  very  difficult.  Only  with  plasma  panel  on  right 
side  panel. 
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11-12.  Can  you  improve  on  the  plasma  panel  arrangement  of  the  digits 
and  the  north,  south,  east,  and  west  symbols? 
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Coenents : 


- Would  have  to  try  several  cowbinatici.s.  Perhaps  enter 
button  should  be  in  tower  left  column. 
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- Is  It  possible  to  take  the  8 spaces  and  make  them  into  6, 
thereby  making  each  digit  larger?  The  enter  key  was  very  hard 
to  reach  no  matter  where  the  location  of  the  keyboard  was. 

- This  type  of  key  placement  would  cut  down  greatly  on  entry 
errors.  You  could  also  enter  numbers  by  “touch"  since  all 
odd  numbers  are  in  the  outside  rows  and  even  numbers  on  the 
inside  rows. 

1/N  P 

2 A 6/E 

3 N 7 

4 E 8 

5/S  L "■  9 

0/W  ENTER 

- Any  layout  isn't  going  to  be  really  good  unless  the  panel  is 
angled  to  deal  with  parallax. 

11-13.  The  legend  on  the  projection  switches  was 

Very  difficult  to  read 

1/9  Moderately  difficult  to  read 

3/9  Moderately  easy  to  read 

5/9  Very  easy  to  read 

Comments: 

- Though  easier  than  any  other  to  read  and  punch*  perhaps  the 
numbers  themselves  could  be  larger. 
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Figure  FI*  Keyboard  legends  on  projection  switches  for  each 
logic  level  step  during  a conrnunication  task. 
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Figure  F2.  Keyboar*d  legends  on  projection  switches  for  each 
logic  level  step  during  a navigation  task. 
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11-14.  How  did  the  size  of  the  switches  and  their  proximity  to  each 

other  on  the  projection  switches  affect  your  ability  to  operate 
the  keyboard  while  wearing  flying  gloves? 

1/9  Made  operation  very  difficult 

Made  operation  moderately  difficult 
3/9  Did  not  affect  operation 
2/9  Made  operation  moderately  easy 
3/9  Made  operation  very  easy 

Comments : 

- Characteristics  of  the  projection  switches  made  operation 
moderately  easy.  No  problems  with  it,  except  on  the  side 
panel.  Then  this  configuration  is  too  long  for  easy  mani- 
pulation. 

11-15.  How  did  the  placement  of  the  digit  "1/N"  on  the  projection 
switches  (second  key  from  left)  affect  your  operation  of  the 
keyboard? 

Made  operation  very  difficult 
1/9  Made  operation  moderately  difficult 
8/9  Did  not  affect  operation 

Made  operation  moderately  easy 
Made  operation  very  easy 
Comments : 

- Placement  of  "1/N"  button  on  projection  switches  made  operation 
moderately  difficult.  Should  be  first  digit. 

- As  with  the  plasma  panel,  you  can  eliminate  the  extra  switches. 

- Placement  of  "1/N"  digit  made  operation  moderately  difficult. 
This  difficulty  would  probably  be  overcome  with  more  use  of 
one  keyboard;  e.g.,  as  you  waald  have  in  an  aircraft  that 
you  are  qualified  in. 
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LUri 


- The  placement  of  "1/N"  on  the  projection  switches  did  not 
affect  operation.  Except  I didn't  like  the  first  blank  key. 


11-16.  Can  you  improve  on  the  projection  switches  arranpenent  of  the 
digits  and  the  north,  south , eas t , an3  west  syr.ibols? 

1/fl  2/S  3/E  4/lJ  5 6 

1 "5  9 0 


Responses: 


eTOT 
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r. 

0 ' 
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2/S 

''T~ 

4/W 

’ 
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Cor.ir-ients: 

- “EMT.ER"  on  the  far  right  riakes  it  loo 

far  aft  vihen  panel  Is 

mounted  on  right  side* 


^ ‘ft  'V  , ' 
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II-17A.  Concerning  the  ease  of  operation  of  the  plasma  panel  and 
projection  switches  when  both  were  located  on  the  front 
panel 


Plasma  panel  was  much  easier  than  the  projection 
switches 

Plasma  panel  slightly  easier  than  the  projection 
switches 

1/9  Plasma  panel  was  equally  easy  as  the  projection 
switches 

2/9  Projection  switches  were  slightly  easier  than  the 
plasma  panel 

6/9  Projection  switches  were  much  easier  than  the  plasma 
panel 

Comments ; 

- Projection  switches  were  much  easier  than  the  plasma  panel. 
Due  simply  to  button  size  and  arrangement. 

• Projection  switches  were  sMghtly  easier  than  the  plasma 
panel,  due  to  aligning  stripes  on  plasma  panel. 

- Projection  switches  were  much  easier  than  the  plasma  panel. 
You  read  what  you  touch  (what  you  see  is  what  you  get)  on 
the  projection  switches. 

- No  question  about  it.  Projection  switches  were  much  easier 
on  the  front  panel  than  the  plasma  panel. 
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II-17B.  Concerning  the  ease  of  operation  of  the  plasma  panel  and 

projection  switches  when  both  were  located  on  the  right 

hand  console 

Plasma  panel  was  much  easier  than  the  projection 
switches 

1/9  Plasma  panel  was  slightly  easier  than  the  projection 
switches 

1/9  Plasma  panel  was  equally  easy  as  the  projection 
switches 

Projection  switches  were  slightly  easier  than  the 
plasma  panel 

7/9  Projection  switches  were  much  easier  than  the  plasma 
panel 

Comments: 

- There  was  a great  deal  of  parallax  in  the  plasma  panel  which 
added  greatly  to  an  already  difficult  situation. 

- Plasma  panel  very  difficult  to  operate.  Pilot's  line  of  sight 
caused  many  erroneous  entry  errors  (one  digit  off).  Lines 
from  readout  to  buttons  deceptive.  If  used  In  this  location, 
panel  should  be  tilted  out  to  a better  reading  angle. 

- Both  gave  me  problems,  but  it  seemed  as  though  I could  tell  at 
a glance.  Plasma  panel  was  slightly  easier  than  the  projection 
switches  on  the  right  hand  console. 

Projection  switches  were  much  easier  than  the  plasma  panel. 

Not  due  to  overwhelming  superiority  of  projection  switches, 
but  due  to  overwhelming  deficiencies  of  plasma  panel  when 
mounted  on  right  side. 

- Projection  switches  were  much  easier  than  the  plasma  panel 
when  located  on  the  right  hand  console.  Due  to  aligning 
stripes  on  plasma  panel.  Plasma  panel  could  possibly  be 
arranged  to  let  the  pilot  look  more  directly  onto  panel. 
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11-18.  Activation  of  the  fourth  logic  level  differs  between 

communication  and  navigation  tasks  on  both  the  plasma  panel 
and  projection  switches.  In  order  to  activate  the  digits 
in  a communication  task,  the  operator  must  select  the  bottom 
right  switch  labeled  "A/N".  For  the  navigation  task,  the 
operator  must  select  the  third  from  the  left  switch  labeled 
"waypoint".  How  did  the  difference  in  the  positioning  of 
the  switch  that  activates  the  digits  affect  your  operation 
of  these  keyboards? 

Made  operation  very  difficult 

3/9  Made  operation  moderately  difficult 

6/9  Did  not  affect  operation 

Made  operation  moderately  easy 

Made  operation  very  easy 

Comments: 

- The  difference  between  keyboards  in  the  positioning  of  the 
switch  that  activates  the  digits  made  operation  moderately 
difficult.  At  first  it  was  difficult  to  go  over  to  A/N, 
but  after  a while  I could  do  it  rather  unconsciously. 

- The  difference  between  keyboards  in  the  positioning  of  the 
switch  that  activates  the  digits  made  operation  moderately 
difficult.  You  should  have  to  remember  only  one  switch 
position  so  that  you  can  go  back  to  habit  patterns  and  not 
have  to  think  about  operation  when  things  are  going  wrong, 
or  when  you  have  to  fly  a difficult  approach  in  weather. 

- A/N  should  not  be  reguired  as  the  computer  can  be  progranned 
to  know  that  digits  will  be  entered. 
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- Would  be  easier  if  the  most  common  use  in  Nav;  i.e.,  "waypoint" 
were  in  the  same  place  as  "A/N"  either  in  the  third  slot  or 
the  last. 

- Placements  of  "A/N"  and  "waypoint"  did  not  affect  operation. 
However,  "A/N"  would  be  easier  to  use  if  positioned  on  same 
side  of  keyboard. 

11-19.  In  regards  to  the  fourth  log^c  step,  how  did  the  placement  of 
the  "enter"  button  in  the  bottom  right  position  on  both  the 
projection  switches  and  plasma  panel  affect  your  operation  of 
the  keyboards? 

Made  operation  very  difficult 
1/9  Made  operation  moderately  difficult 

5/9  Did  not  affect  operation 

2/9  Made  operation  moderately  easy 

1/9  Made  operation  very  easy 

Comments: 

- Position  of  "enter"  did  not  affect  operation.  I would  have 
liked  it  better  in  bottom  left  since  I do  all  operations  left 
handed.  I don't  remove  my  right  hand  from  stick. 

« 

- "ENTER"  on  the  far  right  makes  it  too  far  aft  when  panel  is 
mounted  on  right  side.  Also,  causes  left  hand  work  on  right 
side  of  cockpit  when  mounted  on  front.  Possible  interference 
with  center  stick. 

11-20.  How  did  the  placement  of  the  digits  in  a 789  manner  on  the 
digit  mode  panel  affect  your  456 

operation  of  the  keyboard?  123 

COE 

1/9  Made  operation  very  difficult 
1/9  Made  operation  moderately  difficult 
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3/9  Did  not  affect  operation 
1/9  Made  operation  moderately  easy 
3/9  Made  operation  very  easy 

Comments : 

- Very  good  arrangement.  Since  zero  is  the  most  often  used 
number,  it  should  be  the  closest  and  most  convenient. 

- I had  not  used  such  a configuration  before  and  thus  it  was 
hard  at  first.  During  the  time  I was  flying  the  simulator, 

I purchased  a calculator  with  a similar  keyboard  and  had 

no  problem  from  that  point  on.  I can  see  no  great  problems 
with  it. 

- Placement  made  operation  moderately  easy.  I'm  used  to  using 
an  adding  machine  left  handed  with  the  same  digit  layout. 

- I think  pilots  could  get  used  to  any  pattern. 

11-21.  Figure  G shows  several  digit  arrangements.  The  first  arrange- 
ment was  the  one  you  used  for  the  digit  mode  keyboard  in  the 
experiment.  The  second  and  third  are  like  the  arrangements 
on  most  telephones  and  calculators,  respectively.  We  need 
to  know  whether  any  improvements  need  to  be  made  on  the 
arrangement  of  the  digits  and  the  North,  South,  East,  and  West 
symbols.  Indicate  on  Figure  G which  arrangement  you  like  best 
or  else  write  in  your  suggestion  on  the  blank  arrangement 
provided. 

Comments: 

-MU  JiL.  -3/L 
TTtr 


CLEAR  ^ EHTER 


"gTT-  ~57r 
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Figure  G.  Keyboard  Digit  Arrengements . 
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- I believe  that  any  of  the  arrangements  would  prove  to  be 
feasible.  However,  a distinctive  key  such  as  a bump  or  a 
different  light  could  be  put  on  the  5 for  a starting  point 
so  that  a person  doesn't  have  to  look  down  as  much. 

- Calculator  arrangement  most  desirable. 

- Calculator. 

- Calculator. 

- Digit/mode  keyboard.  I like  this  keyboard  because  "1/N"  etc. 
would  always  be  the  same  on  all  keyboards. 

- Telephone. 

- "Calculator"  arrangement  gives  a more  logical  arrangement  of 
compass  points. 

11-22.  Figures  E,  F,  and  G depict  the  logic  steps  available  on  each 

type  of  keyboard.  Do  you  have  any  additional  comments  in 

regards  to  the  legends  used  or  their  position  on  the  keyboards? 

- The  legends  are  fine,  but  extra  keys  could  be  eliminated  and 
displays  slightly  more  condensed. 

- Plasma  panel  - logic  level  1 should  be  on  the  left  hand  of  the 

panel.  That  would  put  the  first  4 or  5 entires  on  the  left. 

If  both  of  these  panels  were  used  in  the  same  aircraft,  why  not 
arrange  the  labels  in  a similar  pattern  since  each  one  has 
only  two  rows  or  two  columns.  On  any  panel,  some  increased 
accuracy  may  be  available  if  you  put  a plasma  print  out  of  the 
entry  prior  to  “enter"  across  the  top  edge  of  that  panel. 

- Plasma  panel  - Prefer  to  have  logic  level  1 on  column  1 starting 
with  first  button.  Dislike  placement  of  "A/N".  The  most 
important  drawback  to  plasma  panel  is  the  parallax  problems 
involved  in  associating  the  button  with  proper  display. 

- Plasma  panel  - logic  level  1 - place  legends  on  upper  left  4 keys. 

Plasma  panel  - logic  level  4 - do  not  skip  keys.  Same  thing 

for  projection  switches. 
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SECTION  III 

DISPLAY  QUALITY  AND  APPROPRIATENESS 
OF  FORMATS  USED 

The  formats  of  the  Information  presented  on  the  displays  embodied 
one  assessment  of  the  information  requirements  for  this  type  of  mission. 
We  are  interested  in  your  reactions  to  these  formats.  Interest  in  the 
display  format  question  is  two-fold:  (1)  whether  or  not  the  information 

displayed  is  adequate;  and  (2)  wh.ether  or  not  the  format  is  easily 
understood  and  interpreted.  In  addition,  questions  are  included  for  you 
to  evaluate  the  quality  of  the  displays  and  handling  characteristics  of 
the  simulator.  For  each  of  the  following  questions,  fill  in  the  circle 
which  most  nearly  conforms  to  your  opinion.  Figures  H - L depict  some 
of  the  formats  presented  on  the  electro-optical  displays. 

III-l.  On  the  EADI,  the  following  information  was  presented: 

pitch,  bank,  airspeed,  altitude,  heading,  flight  director, 
acceleration,  vertical  velocity,  and  aircraft  symbol.  For 
utilization  during  flight,  this  information  package  was 

1/9  Excessive 

6/9  Sufficient 

2/9  Insufficient 

Comments: 

- Trend  information  on  heading  and  altitude  were  either  missing 
or  hard  to  pick  up. 

- Information  was  adequate,  method  of  presentation  bad. 

- Heading  gives  Insufficient  Information.  A complete 
directional  gyro  Indicator  should  be  Installed  sepantely. 

- EAOI  Information  sufficient.  Everything  other  than 
acceleration  Is  required  for  good  Instrument  crosscheck. 
However,  having  It  on  EAOI  is  as  good  as  any  place  and 
saves  space. 
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- “6"  was  fun  to  watch,  but  not  very  important  for  this 
simulation.  Need  angle  of  attack. 

- Excessive.  "6"  not  used.  Altitude  to  5 digits  is  excessive 
and  distracting  when  at  altitude.  Generally,  3 significant 
digits  is  all  I worry  about. 

III-2.  If  you  found  the  information  displayed  on  the  EADI  excessive, 
which  of  the  following  would  you  eliminate 


Pitch 

2/9 

Heading 

Bank 

Flight  Director 

Airspeed 

3/9 

Acceleration 

Altitude 

1/9 

Vertical  Velocity 
Aircraft  Symbol 

- Heading  and  acceleration  excessive.  Depends  on  rest  of 
displays  available  in  cockpit. 

- I would  like  to  see  the  “G"  meter  and  VVI  exchanged. 

- Eliminate  heading  on  EADI  if  it  is  on  lower  CRT  or  if 
flight  director  is  operative.  The  carrot  (sic)  symbol  of  the 
vertical  velocity  is  too  hard  to  use  with  peripheral 
vision.  Maybe  another  tape  presentation  would  be  better 
next  to  the  altitude  strip. 

1 1 1-3.  If  your  found  the  information  displayed  on  the  EADI  insufficient, 

what  vwuld  you  add? 

- It  might  be  necessary  to  display  the  altimeter  setting 
somewhere.  On  most  aircraft,  mach  nunber  is  not  only 
useful,  but  is  often  a limiting  factor.  On  roost  ADIs,  the 
bank  pointer  is  called  the  sky  pointer  and  is  located  at 
the  top  of  the  AOI.  This  might  cause  some  confusion. 

- When  the  desired  pitch  is  on  or  near  a pitch  index  line, 
and  you  are  centering  the  flight  director  symbol,  precise 
pitch  control  is  not  possible  since  the  symbols  cover  the 
pitch  index. 


« i n,. 
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- More  trend  information  on  heading.  Perphaps  5 numbers 
instead  of  1.  The  VVI  pointer  carrot  (sic)was  very  difficult 
to  read  and  should  be  bigger. 

- Angle  of  attack. 

- Need  trend  information  for  vertical  velocity,  heading,  airspeed, 
altitude.  Find  it  difficult  and  time  consuming  reading 
digital  information  for  airspeed  and  altitude. 

1II-4.  Rate  the  following  information  presented  on  the  EADI  in  regards 
to  the  ease  of  retrieval 


Locating  the  information  was: 


Very 

Moderately 

Moderately 

Very 

Easy 

Easy 

Difficult 

Difficult 

Pitch 

0/9 

1/9 

Bank 

7/9 

1/9 

1/9 

Airspeed 

6/9 

2/9 

1/9 

Altitude 

5/9 

3/9 

1/9 

Heading, 

2/9 

5/9 

2/9 

Flight  Director 

6/9 

3/9 

Acceleration 

3/9 

5/9 

1/9 

Vertical  Velocity 

2/9 

2/9 

4/9 

1/9 

Aircraft  Symbol 

8/9 

1/9 

- Bank  - too  far  away  from  pitch  index.  VVI  - I rarely  used  it. 
Airspeed  and  altitude  - easy  only  if  desired  number  is  at  one 
end  of  the  scale. 

- Need  finer  pitch  scaling, 

- EADI  information  location  excellent. 

- Because  of  the  type  of  flying  that  we  were  doing,  the  heading 
function  did  not  come  Into  ny  crosscheck. 
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- I used  the  altitude  and  airspeed  vertical  bars  almost  entirely 
for  the  testing.  Gives  you  all  your  information  in  one  quick 
glance.  In  turbulence  during  flight,  I think  you  would  have 
difficulty  in  reading  the  small  digits. 

III-5.  Rate  the  following  information  presented  on  the  EADI  according 
to  the  degree  to  which  the  information  was  easy  to  understand 


Understanding  the  information  was: 


Very 

Easy 

Moderately 

Easy 

Moderately 

Difficult 

Very 

Difficult 

Pitch 

7/9 

1/9 

1/9 

Bank 

6/9 

2/9 

1/9 

Airspeed 

6/9 

1/9 

2/9 

Altitude 

3/9 

3/9 

3/9 

Heading 

3/9 

4/9 

1/9 

1/9 

Flight  Director 

7/9 

2/9 

Acceleration 

5/9 

3/9 

1/9 

Vertical  Velocity 

2/9 

4/9 

3/9 

Aircraft  Symbol 

7/9 

1/9 

1/9 

Comments: 

- With  a digital  W that  is  instantaneous,  the  display  changes 
so  rapidly  at  times  that  reading  it  accurately  became  very 
difficult.  I would  also  say  that  the  increasing/decreasing 
sign  is  inadequate.  The  airspeed  indicator  needs  to  be 
calibrated.  The  altitude  trend  indicator  would  be  better 
utilized  if  it  weren't  constantly  switching  from  full  to  zero 
deflection.  Since  most  of  the  flying  is  done  at  altitudes 
such  as  31,000,  24,000,  and  8,000,  I would  suggest  centering 
the  thousand  or  zero  marks  and  scale  it  plus  or  minus  500. 
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- Airspeed  and  altitude  - moderately  difficult.  With  thermometer 
displays  of  this  type,  the  pilot  pays  a penalty  for  being 
accurate.  Unfortunately,  I seldom  had  the  problem  but  those 
who  fly  accurately  would  be  rattled  by  the  tape  changing 
constantly  from  full  to  empty,  to  full...!  Refresh  rate 
seemed  extremely  slow  in  dynamic  maneuvering.  Would  have  to 
see  both  axes  to  evaluate.  What  would  happen  if  steering  was 
bad  in  one  axis? 

- Altitude  - too  many  significant  digits.  Acceleration  and 
VVI  - rarely  looked  at  or  cared.  Understanding  EADI  infor- 
mation excellent. 

- An  adjustable  pitch  trim  knob  is  needed.  The  blue-brown 
horizon  is  excellent  - the  degree  of  bank  would  be  better 
shown  as  a sky  pointer  than  a ground  pointer.  The  moving 
vertical  line  for  airspeed  and  altitude  is  terrible. 

Indications  of  going  "over  the  top"  is  very  confusing. 
Interpreting  rate  from  a vertical  moving  line  is  very 
difficult  and  the  digital  readout  is  too  small  and  difficult 
to  read.  Heading  digital  readout  is  good,  but  too  small. 
Acceleration  is  out  of  field  of  view  and  no  indication  of 
rate.  Almost  unusable.  Vertical  velocity  digits  too  small 
and  rate  indicator  completely  inadequate.  During  rapid  rate 
change,  you  can  go  from  climb  3000  to  descent  3000  without 
showing  the  rate  of  change  passing  zero.  Very  unrealistic. 

- Vertical  velocity  needs  to  be  interpretable  at  a glance,  not 
necessarily  by  reading  it,  The  carrot  (sic)  symbol  was  too  small. 
Attitude  bar  wasn't  the  best  way  to  do  it  eitner.  Seeing  a 

lot  of  bar  didn't  necessarily  mean  that  you  were  way  off 
altitude.  I think  a bar  with  the  desired  altitude  in  the 
middle  with  a visible  teference  line  would  be  better.  Above 
the  reference  you're  high;  below,  you're  low.  Same  for 
airspeed.  Heading  could  have  been  bigger  and  required  too  much 
mental  arithmetic  to  figure  intercepts  and  such.  Didn't  give 
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good  idea  of  rate  of  turn  to  figure  lead  points  for  rolling 
out  on  a desired  heading.  Having  a compass  rose  as  in  a 
normal  HSI  is  very  handy  for  basic  instrument  work.  Excellent 
flight  director.  Needs  angle  of  attack  display. 

- Heading  needs  to  show  a trend  to  help  establish  roll  in/out 
points.  VVI  could  have  larger  carrot  (sic)  symbol. 

- At  a glance  it  was  hard  to  tell  exactly  how  much  bank  you 
were  in.  The  heading  indicator  should  have  trend  information 
on  it.  The  VVI  pointer  carrot  (sic)  should  be  a lot  bigger. 

III-6.  On  the  HSD,  the  following  information  was  presented:  track, 

waypoints,  aircraft  location,  heading,  ground  speed,  and  distance 
to  the  next  waypoint.  For  utilization  during  flight,  this 
information  package  was: 

Excessive 
8/9  Sufficient 
1/9  Insufficient 
Comments:  None 

III-7.  If  you  found  the  information  displayed  on  the  HSD  excessive, 
which  of  the  following  would  you  eliminate: 

Track 
Waypoints 
Aircraft  Location 
1/9  Heading 

Ground  Speed 
Distance  to  Waypoint 

Comments : 

- Distance  to  waypoint  is  sufficient  to  NM  only.  Going  to  feet 
when  close  to  waypoint  not  needed.  Your  position  changes  too 
rapidly  when  flying  6 to  8 miles  a minute.  Feet  information 
not  needed. 
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- Heading  - no  need  for  a redundant  readout,  especially  with 
the  flight  director  working. 

I I 1-8.  If  you  found  the  information  displayed  on  the  ^ insufficient, 
what  would  you  add? 

- Had  some  problem  remaining  oriented  to  the  real  world.  An 
improved  heading  display  on  the  EADI  or  HSO  may  help  and  a 
feature  should  be  added  to  allow  the  pilot  to  switch  to  north 
up  display. 

- Digits  were  sometimes  too  small  on  the  green  light  scope. 

One  time  I mistook  a 1 for  a 2.  I wanted  magnetic  course 
and  length  of  each  leg  printed  next  to  the  leg  I was  on. 

- Somewhere  in  the  cockpit  a compass  rose  is  needed. 

- Time  to  waypoint  might  be  another  consideration. 

III-9.  Rate  the  following  information  ,..2sented  on  the  H^  in  regards 
to  the  ease  of  retrieval. 

Locating  the  information  was: 

Very 
Easy 

Track  9/9 

Waypoints  9/9 

Aircraft  Location  7/9 

Heading  5/9 

Ground  Speed  7/9 

Distance  to  Waypoint  7/9 

Conments: 

- Move  ground  speed  and  distance  to  waypoint  toward  center. 

Why  use  corners.  It  expands  necessary  crosscheck  area. 

Digits  should  also  be  larger. 


Moderately  Moderately  Very 

Easy  Difficult  Difficult 


1/9 

4/9 

2/9 

2/9 


1/9 
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- In  regards  to  your  normal  heading  indicator,  much  more 
convenient  to  have  a complete  compass  card. 

III-IO.  Rate  the  following  information  presented  on  the  according 
to  the  degree  to  which  the  information  was  easy  to  understand. 

Understanding  the  information  was: 


Very 

Moderately 

Moderately 

Very 

Easy 

Easy 

Difficult 

Difficult 

Track 

9/9 

Waypoints 

9/9 

Aircraft  Location 

7/9 

1/9 

1/9 

Heading 

4/9 

3/9 

1/9 

1/9 

Ground  Speed 

7/9 

2/9 

Distance  to  Waypt. 

7/9 

2/9 

Comments: 

- There  must  be  a better  symbol  for  an  aircraft  than  an  isoscles 
triangle.  That  made  it  difficult  to  interpret  heading  even 
when  you  knew  you  were  on  course.  How  about  when  you  find 
yourself  in  an  unusual  altitude  and  lost. 

- I would  usually  prefer  to  use  a proportion  of  the  known  leg 
length  to  deter;inine  distance  to  next  waypoint,  unless  precise 
number  was  needed. 

- Heading  and  aircraft  location  moderately  difficult  In  relation 
to  a compass  because  you're  just  looking  at  one  heading 
number. 

- Heading  - very  difficult  to  understand  Information.  I knew 
where  to  find,  but  I couldn't  use  It. 

- Ground  speed  and  distance  Indicators  could  have  been  closer 
to  the  display. 
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Ill-n.  In  reference  to  the  legs  of  the  mission  track  presented  on  the 
HSD,  the  display  changed  to  a jagged  line  when  the  aircraft  was 
off  the  specific  heading.  How  did  this  presentation  affect 
the  tracking  task? 

1/9  Made  the  tracking  task  much  easier 

2/9  Made  the  tracking  task  somewhat  easier 

5/9  Did  not  affect  the  tracking  task 
1/9  Made  the  tracking  task  somewhat  difficult 
Made  the  tracking  task  much  more  difficult 

Comments : 

- A nice  convenience,  but  not  necessary.  Flying  involves  a 
continual  change  of  headings  and  power  settings  to  maintain 
desired  track  and  altitude. 

- This  feature  made  the  tracking  task  somewhat  easier.  It  was 
the  only  way  I could  get  a feel  for  how  my  heading  varied 
from  the  course  line,  since  you  didn't  tell  me  what  the  course 
really  was.  Even  with  the  known  course,  this  is  not  easy  due 

to  relative  heading  (without  some  type  ‘of  vertical  grid  system), 

“ This  feature  did  not  affect  the  tracking  task.  Sorry,  but  I 
seldom  tried  to  obtain  perfection  due  to  control  response  and 
sensitivity,  and  associated  tendency  to  over  control. 
Accordingly,  a "close  enough  for  government  work"  attitude 
set  in,  particularly  during  switching  tasks.  If  the  bank 
command  was  close  to  center  and  aircraft  was  on  track,  I 
didn't  care  whether  the  line  was  smooth  or  Jagged. 

Rate  the  following  display  qualities  according  to  the  scales 
provided  and  make  a mark  in  the  appropriate  box.  When  making  additional 
comments.  Include  the  number  referring  to  the  specific  display 
quality. 
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III-12.  EADI 


(Fig  H) 


Display  Qualities 


Very  Un- 

Unaccept- 

No 

Accept- 

Very 

acceptable 

able 

Opinion 

able 

Acceptable 

1. 

Legi- 

bility 

2/9 

4/9 

3/9 

2. 

Bright- 

ness 

4/9 

5/9 

3. 

Color 

4/9 

5/9 

4. 

Shape  of 
Symbol s 

1/9 

4/9 

4/9 

5. 

Jitter 

1/9 

2/9 

4/9 

2/9 

Comments : 

- I 

like  having 

the  VVI  below  the  altimeter  and 

in  Instrument 

approaches  it  is  important.  On  the  airspeed  and  altitude 
displays,  I would  prefer  removal  of  the  letters  and  making 
the  numbers  larger,  especially  the  altimeter.  The  trend  bars 
on  either  side  are  quite  useful,  but  I don't  really  think 
they  need  to  be  quite  so  large.  The  aircraft  symbol  center 
dot  needs  to  be  more  distinct  because  when  its  superimposed 
on  the  flight  director,  you  lose  positive  control  of  aircraft 
and  have  to  resort  to  altimeter  and  VVI  since  you  cannot  see 
exactly  what  pitch  the  airplane  is  at. 

- Legibility  - in  rough  air  flying,  it  would  be  difficult  to 
read  the  small  number  displays.  Therefore,  the  vertical 
bars  are  a great  aid. 

- Jitter  - it  sometimes  gave  me  a distortion  of  view,  as  though 
the  lines  were  floating. 

- Legibility  - unacceptable.  Find  it  difficult  and  time  con- 
suming reading  digital  information  for  airspeed  and  altitude. 


•♦•a*  V 
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Shapes  of  symbols  - unacceptable.  Basic  aircraft  symbols 
are  probably  ok,  perhaps  they  should  be  finer.  Other  displays 
don't  like.  Jitter  - very  unacceptable.  When  attitude  was 
changing. 

- Jitter  somewhat  distracting  at  first,  (acceptable) 

- Can  color  change  when  there  are  large  deviations  in  VVI, 
airspeed,  and  "Gs“?  Also,  intensity  change  at  night? 

“ Brightness  and  color  very  acceptable.  Great  now.  How 
about  after  CRT  gets  old?  Will  they  fade? 


HSD 

(Fig  I) 

Display  Qualities 

Very  Un-  Unaccept- 
acceptable  able 

No 

Opinion 

Accept 

able 

- Very 
Acceptable 

1.  Legi- 
bility 

1/9 

5/9 

3/9 

2.  Bright- 
ness 

5/9 

4/9 

3.  Color 

5/9 

4/9 

4.  Shape  of 
Symbols 

5/9 

4/9 

5.  Jitter 

1/9 

6/9 

2/9 

Comments: 

Legibility  of  digits  unacceptable. 


- Jitter  - very  unacceptable.  Sometimes  parts  of  the  display 
would  disappear  for  a noticeable  period  of  time. 

- You  are  not  concentrating  on  the  HSD  as  intently  as  the  EAOl, 
so  the  Jitter  is  not  noticeable. 


I 
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- A compass  rose  is  needed  for  reference.  It  would  also  help 
if  the  pilot  could  annotate  how  the  waypoints  correlate  to 
outside  reality. 

III-14.  Certain  information  pertaining  to  communication  and  navigation 
was  presented  on  the  MPD.  In  your  opinion,  for  each  of  the  two 
formats,  was  this  information: 


Comm 

Nav 

2/8  * 

2/8 

Excessive  for  utilization  during  flight 

6/8 

6/8 

Sufficient  for  utilization  during  flight 

Insufficient  for  utilization  during  flight 
* One  subject  did  not  indicate  a response. 


Comments: 

- Too  many  systems  listed  in  both  "comm"  and  "nav"  displays. 
Too  cluttered.  You  need  only  to  know  whether  a system  is 
"on"  or  "off."  Therefore,  display  only  what  is  "on"  be- 
cause you  know  what  you're  using.  In  the  same  regard.  If 
it  doesn't  appear  on  the  MPD,  you'll  know  it's  "off." 

- Because  of  the  type  of  mission  we're  flying,  tended  not  to 
use  this  information. 

- Never  really  paid  much  attention  to  NAV  status. 

- I really  didn't  spend  enough  time  looking  at  the  Information 
on  the  MPD  to  evaluate  it. 
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NAV  STATUS 


BEARING 

DIST 

346 

800F 

FROM 

TO 

T.  HEADING 

P.P. 

3 

359 

TIME  TO  SELECTED  WAYPOINT 

0 HRS 

0*  20" 

6N0  SPO 

DRIFT  ANGLE 

53  KTS 

OOOR 

NAV.  MODE 

TCN 

SAT  SCALE 

D. I . S 1 

126 

26  40n 

PRESENT  POSITION  WAYPOINT 


N 15  42*  06" 

E 108  15*  03" 


Figure  J.  Multipurpose  display  fornat  for  navigation 
status. 
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COMM  STATUS 

SUBSYSTEM 

PWR 

CHAN 

FREQ 

CMD  UHF 

OFF 

1 

335.05 

vhf/fm 

OFF 

2 

XXX. XX 

ADF/AUX  UHF 

ON 

3 

XXX. XX 

MODE 

PWR 

CODE 

IFF 

OFF 

1 

ON 

005 

2 

ON 

201 

3/a 

OFF 

0000 

c 

ON 

156 

4 

OFF 

304 

RADAR  BCN 

ON 

929 

Figure  K.  Multipurpose  display  format  for  communication  status. 


122 


^ -• 


AFFDL-TR-77-9 


I I 1-1 5.  Interpretation  of  the  information  on  the  1^  pertaining  to 

communication  and  navigation  was: 

Comm  Nay 

1/8  * 1/8  Very  easy 

6/8  6/8  Moderately  easy 

1/8  1/8  Moderately  difficult 

Very  difficult 

*0ne  subject  did  not  indicate  a response. 

Comments ; 

- I had  to  continually  search  for  things  because  tasks  did  not 
involve  much  use  of  this  information. 

• Some  NAV  data  could  be  put  on  HSD.  I think  the  digits  are 
too  small  compared  to  the  size  of  other  symbols  in  the 
cockpit. 

- Too  cluttered.  Until  you  were  used  to  the  position  of  the 
various  readouts,  you'd  have  to  go  through  the  entire  list 
of  displayed  material  to  find  what  you  wanted  to  know. 

- left  Mt*0  was  too  cluttered  and  printing  was  too  small.  You 
had  to  search  for  information  - distracting  from  primary 
task. 

• NPO  pre-entry  readouts  should  be  larger  for  easier  use  during 
cross  checks. 
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1 11-16.  Interpretation  of  the  information  on  the  MPD  pertaining  to 
failure  warnings  was: 

5/7*  Very  easy 

1/7  Moderately  easy 

1/7  Moderately  difficult 

Very  difficult 

*Two  subjects  did  not  indicate  a response. 


Comments: 

- I think  the  digits  are  too  small  compared  to  the  size  of 
other  symbols  in  the  cockpit. 

- Lettering  should  be  larger. 

- Never  used  because  it  was  unnecessary,  since  you  only  had  to 
be  aware  of  the  lighted  panels  in  the  problems  presented. 

- A nice  convenience,  but  you  need  only  to  know  what  panel  you 
should  be  working  on. 

- Master  caution  light  was  good.  Printing,  again  was  too  small 
and  hard  to  read. 

- Perhaps  in  addition  to  failure  warnings,  you  could  have  a 
checklist  page  to  refer  to  on  the  display. 

- Possibly  unnecessary  as  the  proper  panel  to  use  was  illuminated 
for  each  large  logic  level  stop. 
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FP  KEYBOARD  FAIL 
USE  RC  FOR  2,  3 


Figure  1.  Multipurpose  display  foriet  for  keyboerd  failure  warnings. 
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Even  though  the  simulator  flying  qualities  were  designed  to  keep 
you  busy,  we  are  still  interested  in  rating  the  simulator.  Rate  the 
following  simulator  qualities  according  to  the  scales  provided  and  make 
a mark  in  the  appropriate  box.  When  making  additional  comments,  include 
the  number  referring  to  the  specific  simulator  quality. 

III-17.  Simulator  Qualities 

The  performance  of  the  simulator  can  be  broken  down  into  the 

following  qualities: 

(1)  The  degree  to  which  the  electronically  generated  symbology 
on  the  displays  moves  ciynamically  in  real  time  in  RESPONSE 
to  the  action  of  the  control  stick.  This  response  quality 
refers  specifically  to  the  displayed  pitch,  bank,  flight 
director,  track,  and  waypoints. 

(2)  The  degree  to  which  the  dynamic  response  of  the  displays 
to  the  control  stick  action  was  SMOOTH  in  contrast  to 
abrupt. 

(3)  The  degree  to  which  the  symbology  on  the  displays  follows 
the  appropriate  DIRECTION  as  determined  by  the  control 
stick  action. 

(4)  The  degree  to  which  displayed  flight  parameters  (airspeed, 
altitude,  heading,  acceleration,  bearing,  and  ground 
speed)  are  REALISTIC  INDICATIONS  of  the  aircraft's  state 
in  the  particular  portion  of  the  mission. 

Very  Un-  Unaccept-  No  Accept-  Very 
acceptable  able  Opinion  able  Acceptable 


1. 

Response 

2/9 

2/9 

5/9 

2. 

Smoothness 

1/9 

6/9 

2/9 

3. 

Direction 

5/9 

4/9 

4. 

Indicators 

1/9 

1/9 

4/9 

3/9 

... 
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Comments: 

- Response  unacceptable.  Much  too  sensitive.  Smoothness 
acceptable.  Difficult  to  be  absolutely  sure  without  motion. 
Indicators  unacceptable.  The  airspeed  was  tied  too  closely 
with  throttle  and  not  enough  with  pitch.  The  vertical  velocity 
was  tied  too  closely  with  pitch  and  not  enough  with  throttle. 

- Indicators  acceptable.  Tapes  might  be  more  useful  if 
desired  level  were  set  as  an  index  in  the  middle  of  the  tape. 

- Stick  has  slightly  too  much  centering  force.  Feels  too 
spring  loaded.  Side  stick  is  canted  too  far  back  requiring 
operation  by  middle  two  fingers  and  thumb,  which  is  usually 
on  trim  or  mike  buttons.  Inadvertent  operation  of  these 
buttons  is  too  easy  and  stick  is  not  very  comfortable. 

- The  vertical  velocity  was  unrealistic.  It  very  rarely 
indicated  “O".  During  flight  with  smooth  control  inputs,  it 
could  vary  from  500  climb  to  500  descent  continuously, 

- Very  responsive! 

- Smoothness  acceptable.  Sometime  you  had  to  be  a little  jerky 
on  the  control  stick  to  get  the  display  to  move,  instead  of 
applying  gentle,  smooth  control  pressure  on  the  stick, 
especially  with  the  side  stick.  This  occurred  primarily  when 
it  was  well  trimmed.  Also,  the  airspeed  did  not  fall  off  as 
rapidly  as  it  would  in  real  life  after  rolling  into  60-90 
degrees  bank  with  no  power  increase. 

- Indicators  unacceptable.  No  real  trends  as  to  heading  and 
changes  in  airspeed  or  altitude. 
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Use  this  page  for  any  additional  comments  you  might  have 
concerning  this  multifunction  keyboard  evaluation. 

- I personally  always  fly  your  type  cockpit  with  may  right  hand 
and  do  tasks  with  n\y  left.  On  your  numeric  displays,  a single 
digit  clear  function  would  be  the  best  thing  you  could  install. 

- Possibly  if  the  entire  system  could  be  incorporated  into  a 
simulator  with  motion,  a better  evaluation  could  be  given. 
Overall,  I thought  the  system  was  excellent  and  probably  the 
only  exception  would  be  the  plasma  panel  and  I would  recommend 
that  if  at  all  possible  it  be  eliminated  and  replaced,  or 
improved  upon  as  previously  suggested. 

- I left  off  the  heading  display.  The  pilot  should  have  a 
separate  directional  gyro.  Heading  is  simply  not  enough. 

The  best  indicators  on  the  EADI  as  far  as  "fast  scan" 
instruments  are  the  altitude  and  speed  bars.  Two  excellent 
indicators!  They  give  your  trend  indications  plus  readout. 

By  "trend"  I mean  speed  at  which  you  are  departing  or  getting 
to  your  assigned  altitude  or  airspeed. 

- On  the  EAOI,  1 might  reorganize  the  display  to  place  related 
information  in  close  proximity;  i.e.,  the  altitude,  and  VVI 
setting  on  the  right,  mach  number  and  airspeed  on  the  left. 

I would  also  place  the  bank  indicator  close  to  a heading 
display. 


AFFDL-TR-77-9 


APPENDIX  F 

STATISTICAL  PROCEDURES  USED  IN  DATA  ANALYSES 

Amplitude  distributions  (Reference  FI)  of  the  time-history  recordings 
of  each  parameter  were  constructed  to  evaluate  the  relative  effects  of  the 
experimental  conditions.  Summary  statistics  descriptive  of  the  error 
amplitude  distribution  of  a sample  of  tracking  performance  were  computed 
using  the  following  formulae: 

AE  (Average  error)  « 

T /o  ® 

AAE  (Average  absolute  error)  = 

1 ri 

T Jo  |e  (t)l  dt 

RflS  (Root-nean-square  error)  = 

]/\  fl  w « 

SD  (Standard  deviation)  " 

y/msf  - (AE)^ 

where  T is  the  time  over  which  the  parameter  was  integrated,  e is  the 
amplitude  of  the  parameter  at  time  t,  and  dt  is  the  sampling  interval. 

The  AE  is  a numerical  index  of  the  central  tendency  of  the  amplitude 
distribution,  while  the  SD  reflects  the  variability  or  dispersion  of  the 
measures  around  this  central  tendency.  RMS  error  is  also  an  index  of 
performance  variability,  but  relative  to  the  null  point  rather  than  the 
AE.  AAE  is  the  mean  of  the  amplitude  distribution  replotted  with  all 
error  amplitudes  positive  and  is  indicative  of  the  variability  when 
interpreted  in  conjunction  with  the  other  performance  indices. 

These  summary  statistics  (AE,  AAE,  RMS,  SD)  were  computed  on  the 
flight  parameters  ground  speed,  altitude,  and  flight  director  deviation 
from  null  for  the  time  period  specified  by  the  event  and  for  the  inmediate 
fifteen  seconds  prior  to  the  event.  Summary  statistics  for  the  fifteen 
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second  pre-event  time  for  each  parameter  were  subtracted  from  the 
corresponding  values  computed  for  the  event  In  order  to  measure  only  the 
effect  of  the  keyboard  operations  on  the  pilot's  performance.  An 
example  calculation  can  be  Illustrated  as  follows: 


Altitude  AAE 
Event  Time 


Altitude  AAE 
Pre-event  Time 
(15  sec.) 


Delta  Altitude  AAE 
Summary  Statistic 
Used  In  Statistical 
Analysis 


Keyboard  task  performance  was  evaluated  by  measuring  the  time 
required  for  the  event  and  the  number  of  switch  hits.  Since  the  number 
of  required  switch  hits  was  not  the  same  for  the  communication  changes 
and  navigation  updates,  a Figure  of  Merit  (FOM)  was  computed  by  dividing 
the  actual  number  of  switch  hits  by  the  number  required  to  accomplish 
the  keyboard  operation  without  error.  For  example,  19  switch  hits  were 
required  to  complete  a navigation  update  correctly.  Suppose  a pilot 
made  an  error  on  the  fourth  switch  hit,  cleared  the  entry,  and  then 
entered  the  entire  update  correctly.  The  pilot  then  actually  made  24 
switch  hits  Including  the  clear  button  In  order  to  successfully  complete 
the  update.  The  FOM  In  tuis  case  would  be: 

24  (Actual  number  of  switch  lilts  made) 

15  (Number  of  switch  hits  required  to  complete  » 1,26 

task  without  error) 

The  communication  change,  on  the  other  hand,  required  nine  switch  hits. 
Let's  say  the  pilot  selected  three  digits  before  realizing  he  made  an 
error,  cleared  the  entry,  and  then  selected  and  entered  the  correct 
frequency*  The  FON  In  this  casa  would  be: 

13  (Switch  hits  made)  . i aa 

9 (Switch  hHs  required) 

An  error  free  task  would  produce  a FON  of  l.D  and  as  errors  Increased,  the 
FOM  Increased. 
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Statistical  analyses  were  conducted  on  the  following  dependent 
variables: 


- delta  altitude  AAE 

- delta  altitude  RMS 

- delta  ground  speed  AAE 

- delta  ground  speed  RMS 

- delta  bank  AAE 

- delta  bank  RMS 

- keyboard  operation  time 

- figure  of  merit 

These  variables  were  initially  analyzed  by  the  use  of  the  Biomedical  (BMD) 
Statistical  Computer  Program  12V  (Reference  F2)  which  performs  multivariate 
analysis  of  variance  or  covariance  for  any  hierarchical  design  with 
equal  cell  sizes.  In  those  cases  where  the  MANOVA  revealed  significant 
effects,  stepwise  discriminant  function  analyses  by  the  use  of  the 
BMD07M  program  were  conducted.  In  performing  a multiple  group  discriminant 
analysis,  this  program  computes  a set  of  linear  classification  functions 
by  choosing  the  independent  variables  in  stepwise  manner.  The  variable 
entered  at  each  step  is  selected  by  one  of  four  available  criteria 
and  a variable  is  deleted  when  Its  F-value  becomes  too  low. 


M.  Obermayer,  R.  W.,  and  Muckier,  F*  A.,  "Performance  Measurement  In 
night  Simulation  Studies."  NASA-CR-82,  July,  1964. 

F2.  Dixon,  W.  J.,  Ed.,  "BMD  Biomedical  Computer  Programs."  University 
of  California  Press,  California,  1974. 
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APPENDIX  G 
WORK  ANALYSIS 

An  analysis  was  performed  in  order  to  provide  additional  information 
about  the  total  effect  of  different  keyboards  as  a function  of  all  of  the 
dependent  variables.  Although  the  dependent  variable  most  sensitive  to 
changes  in  the  independent  variables  could  be  identified,  there  was  no 
consistent  pattern.  The  total  differences  caused  by  variation  in  the 
independent  variable  could  be  measured  only  by  a combination  of  the  measure 
ments  of  the  dependent  variable.  A single  metric  for  task  difficulty  was 
desired.  This  task  difficulty  is  defined  as  "work"  for  this  analysis. 

The  authors  realize  that  true  work  measurements  would  include 
physiological  factors  and  mental  stresses,  and  that  true  work  could  vary 
without  changes  in  the  parameters  which  were  measured.  For  simplicity 
of  discussion,  the  term  "work"  is  used  as  a measure  of  task  difficulty. 

Easy  tasks  require  a small  amount  of  work.  Difficult  tasks  require  more 
work.  A necessary  assumption  for  the  following  analysis  is  that  the 
subjects  were  actively  Involved  In  the  task.  Observations  by  the  experi- 
menters and  discussions  with  the  subjects  after  the  experiment  confirmed 
this  assumption. 

It  was  hypothesized  that  total  work  (or  task  difficulty)  was  a com- 
bination of  work  for  the  flying  task,  plus  work  for  the  keyboard  task. 

W = + W,^  (Gl) 

As  mentioned  In  Paragraph  III-4.4,  It  was  assumed  that  the  realism  of 
the  simulation  and  the  difficulty  of  the  flying  task  would  establish  a 
work  demand  sufficiently  difficult  that  perfect  performance  o^  the  flying 
task  was  not  possible  and  Imposition  of  a keyboard  task  would  result  In 
a further  degradation  of  flying  task  performance  parameters.  Variations 
In  keyboard  task  difficulty  would  also  be  reflected  In  the  keyboard  task 
parameters;  errors  and  time  to  correct  completion.  The  results  obtained 
through  testing  have  verified  these  assumptions.  The  difficulty  of  the 
flying  task  at  any  given  moment  was  reflected  not  only  In  the  average 
error  from  each  performance  parameter  but  also  In  the  variation  of  this 
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error.  With  no  keyboard  task,  this  work  level  would  be  expected  to  remain 
essentially  constant.  When  increased  attention  was  given  to  the  keyboard 
task,  the  average  errors  in  the  flying  task  performance  parameters  increased 
and  the  variations  in  the  errors  became  greater. 

RMS  of  the  error  is  sensitive  to  changes  in  either  the  magnitude  of 
the  error  or  the  variations  in  the  error,  or  both.  For  this  reason,  RMS 
was  selected  as  the  appropriate  statistic  to  indicate  changes  in  the 
workload  reflected  in  the  performance  of  the  three-dimensional  tracking 
task.  A linear  combination  of  ground  speed,  altitude,  and  flight  director 
bank  error  RMS  was  used  to  measure  the  work  associated  with  the  three 
flying  task  performance  parameters.  Since  these  errors  were  measured 
twenty  times  a second,  they  constitute  error  rates  or  errors  per  0.05 
second. 

Flying  an  airplane  is  a continuous  task.  Work  continues  at  a more 
or  less  constant  rate  and  the  total  amount  of  work  increases  with  time. 

Thus,  the  linear  combination  of  parameters  involves  a multiplication  of 
error  rate  by  time  in  order  to  get  the  cumulative  (or  total)  work  measured 
by  flying  task  performance  parameters. 

Wf  » b^  At  + bg  Vt  + b3  Bt  (G2) 

Where  b's  are  the  weighting  coefficients,  A is 
altitude  RMS,  V is  velocity  (OS)  RMS,  and  B is 
flight  director  (bank)  RMS,  is  work  on  the 
flying  task,  and  t is  the  time  interval  during 
which  work  was  done. 

The  weighting  coefficients  can  be  expected  to  vary  with  differences 
in  flight  control  systems.  For  this  experiment,  control  laws  were  not 
varied  and  the  coefficient  remained  constant.  Indications  of  keyboard 
task  difficulty  were  obtained  by  measuring  the  time  required  to  complete 
the  task  and  computing  the  Figure  of  Merit  (errors  made  during  the  task). 
Difficult  (high  workload)  tasks  would  be  expected  to  take  more  time  and/or 
produce  more  errors. 


^k  * ^4^  * ^5^ 


(S3) 
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Where  b's  are  weighting  coefficients,  t is  time, 

F is  Figure  of  Merit,  and  W is  work  on  the 
keyboard  task. 

When  Equations  G2  and  63  are  substituted  into  Equation  61,  the 
result  is: 


W = b^  At  + b2Vt  + b3Bt  + b^t  + b^F  (64) 

The  five  parameters  in  Equation  64  are  indicated  in  Table  1.  Each 

pilot  was  assigned  the  same  original  tasks.  If  each  pilot  completed  the 

keyboard  tasks  without  error,  the  work  (as  measured  by  performance  indices) 

accomplished  by  each  pilot  would  be  the  same.  However,  since  the  pilots 

were  required  to  complete  the  task  correctly,  errors  on  the  keyboard  task 

resulted  in  a requirement  to  do  additional  work  to  correct  the  performance. 

This  additional  work  is  reflected  in  more  switch  hits  and  is  included  in 

the  Figure  of  Merit  parameter.  Recall  that  the  Figure  of  Merit  is  the 

number  of  switch  hits  used  for  the  task  divided  by  the  minimum  number  of 

switch  hits  that  should  be  needed  to  complete  the  task.  When  the  parametric 

terms  in  the  work  equation  are  divided  by  the  Figure  of  Merit,  the  work 

term  no  longer  represents  the  total  work  done  by  a pilot  but  the  work  done 

"per  switch  hit."  This  is  a form  of  nojinal ization  that  allows  comparison 

of  work  despite  the  fact  that  the  switch  hits  (and  hence,  total  work)  vary 

with  the  task  because  of  task  differences  or  because  of  operator  errors. 

When  both  sides  of  Equation  64  are  divided  by  F and  the  constant  br  is 

combined  with  the  work  term  so  that  W„  » W - br,  W^ 

0 p 5 0 

represents  the  work  required  to  fly  the  simulator  and  actuate  a switch, 
and  constitutes  a standard  task.  Then 


W-  “ b.  At  + b,  Vt  + b,  Bt  + b,  t 

0 Ip  if  Jp 


(G5) 


It  should  be  apparent  that  the  W^  for  any  subject  is  the  same  as  W^^  for 
any  other  subject.  This  equality  will  remain  valid  despite  the  use  of 
various  strategies  by  the  subjects.  That  is,  given  a standard  task  (W^), 
its  work  value  remains  constant  regardless  of  which  parameter  receives 


•H'  * 
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the  greatest  share  of  operator  attention.  This  conclusion  is  true  if  and 
only  if  the  flying  task  is  sufficiently  difficult  that  it  requires  the  full 
attention  of  the  operator.  As  was  discussed  above,  the  condition  was  met 
in  this  experiment. 

In  Equation  65,  the  values  of  the  four  weighting  coefficients  (b^) 
are  unknown,  but  there  are  nine  equations  of  the  same  form  as  65;  (one 
for  each  pilot  based  on  his  average  performance  during  normal  tasks). 

This  constitutes  an  over  specified  set  of  equations. 

A standard  program  (8MD02R)  was  used  to  obtain  a solution  from  the  over 
specified  set  of  equations,  based  upon  performances  of  the  nine  subject 
pilots  on  normal  communication  and  navigation  tasks.  To  obtain  the  weighting 
coefficients,  stepwise  regressions  of  the  dependent  variables  were  performed, 
using  each  variable  in  turn  as  the  initial  criterion,  and  the  others  as 
predictors.  The  value  of  was  arbitrarily  set  at  100.  Of  the  various 
linear  solutions  obtained,  the  following  best  fit  equation  was  selected: 

100  = 2.25150  t + 0.00196  At  + 0.03758  Vt  + 0.25079  Bt  (66) 

f T T r 

It  was  selected  because  all  coefficients  of  factors  contributing  to 
work  are  positive,  reflecting  the  fact  that  increases  in  errors  and  time 
are  indicative  of  increased  work.  Note  that  by  setting  the  constant  at 
100  for  this  "standard"  work  task,  solutions  to  the  equation  based  upon 
other  tasks  will  be  in  terms  of  percentage  of  the  standard  task.  Thus, 
the  equation  establishes  a metric  for  task  difficulty,  based  upon  flying 
task  errors,  keyboard  task  errors,  and  time  to  accomplish  the  keyboard 
task.  An  internal  check  of  the  equation  reveals  that  an  altitude  error 
of  100  feet  is  equivalent  to  a velocity  error  of  five  knots.  Flying 
experience  establishes  that  holding  velocity  within  five  knots  is  about 
the  same  order  of  difficulty  as  holding  altitude  within  100  feet.  It 
should  be  noted  that  Equation  65  is  sufficiently  general  to  be  used  for 
evaluation  of  keyboard  task  performance  for  any  aircraft.  Equation  66 
has  weighting  coefficients  that  are  specific  for  this  simulation. 

Because  the  coefficients  will  vary  with  changes  in  the  flight  control 
systems,  it  is  deemed  inappropriate  to  use  this  method  for  comparing 
flight  control  systems. 
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